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EXECUTIVE SUMMARY 


AN EVALUATION OF THE CULTURAL CONTEXTUALIZATION OF THE 
INCREDIBLE YEARS PARENTING PROGRAM FOR AFRICAN AMERICAN AND 
HMONG CULTURAL GROUPS 


This report is based upon work supported by the Social Innovation Fund (SIF), a key White House 
initiative and program of the Corporation for National and Community Service (CNCS). Greater Twin 
Cities United Way received funds from the SIF to serve as a grantmaking intermediary in Minneapolis 
and Saint Paul. As part of a competitive selection process, the Amherst H. Wilder Foundation (Wilder) 
was selected as the fiscal agent for the Saint Paul Promise Neighborhood’s (SPPN) subgrant. 

Through the SIF award, Wilder and SPPN intended to deliver culturally contextualized versions of the 
Incredible Years program to African American and Hmong parents and children in the SPPN. 


The Center for Applied Research and Educational Improvement (CARED) was awarded the contract to 
evaluate several of the seven nonprofit subgrantee programs that received SIF funding from the United 
Way. This evaluation report provides information about the cultural contextualization process and the 
effectiveness of the culturally-contextualized Incredible Years parent programs, specifically for African 
American and Hmong parents and children. The subgrant period reported here ran from April 15, 2013, 
through December 31, 2015, which includes Year 1 (April 2013-April 2014), Year 2 (April 2014-April 
2015), and a Year 2 no-cost extension (April 2015-December 2015). 


SPPN is part of the U.S. Department of Education’s Promise Neighborhood network. The SPPN, located 
in the Frogtown and Summit-University neighborhoods of St. Paul, has large African American and 
Hmong communities; however, a study by Wilder found low participation in early learning settings on the 
part of African American and Hmong families. The Wilder and SPPN’s theory of change hypothesizes 
that “cultural engagement is a critical piece of the education puzzle” and states that “all students and 
families have access to educational opportunities that honor and fully integrate their cultural practices, 
values, communication preferences, and learning styles” (Saint Paul Promise Neighborhood, 2015). With 
the SIF subgrant, SPPN and Wilder proposed to develop an African American contextualization of the 
Incredible Years parenting program, to refine a Hmong contextualization of the program, and to measure 
the effectiveness of the contextualizations on parenting skills and children’s social-emotional 
development. Incredible Years is an evidence-based program that emphasizes a strong parent-child 
relationship and enhancement of parenting skills. Each participating parent(s) is asked to select a focal or 
“interactive” child for the program. The interactive child joins the parent at the end of each session to 
work on techniques that were taught to the parent. The program provides childcare for the interactive 
child and siblings during each session. 


Prior to applying for SIF funding, Wilder worked with a University of Minnesota professor to adapt the 
Incredible Years model to be culturally-specific and relevant for Hmong parents and children. As part of 
the SIF funding, SPPN and Wilder wanted to further test this cultural adaptation for Hmong families, as 
well as develop and study a cultural adaptation for African American parents and children. 


The evaluation study was designed to describe the implementation of the contextualized versions of the 
Incredible Years parenting program and to measure the effect of the cultural contextualization on 
outcomes of African Americans and Hmong Americans. Because the Hmong contextualization was early 
in the testing stage and the African American contextualization had not yet been developed at the start of 
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the subgrant period, the evaluation team determined that it was appropriate and feasible to obtain a 
preliminary level of evidence for each of these contextualizations by Year 2. 


The impact evaluation questions were: 

e Do parents who participate in the culturally contextualized versions of Incredible Years show a 
reduction in parental stress and/or an increase in effective parenting skills, as measured by the 
Parenting Stress Index-Short Form (PSI-SF; Abidin, 1995)? 

e Dochildren participating in the culturally contextualized versions of Incredible Years show 
improvements in social-emotional behaviors, such as emotional problems, conduct problems, 
hyperactivity, peer problems, and prosocial behavior as measured by the Strengths and 
Difficulties Questionnaire (SDQ; Goodman, 1997)? 


The implementation evaluation questions were: 

e Are parents and children receiving the designated amount of treatment programming? 

e Is the Incredible Years Program being implemented as intended? 

e Are the culturally contextualized versions of the Incredible Years model incorporating all of the 
key elements of the standard Incredible Years model (i.e., is the program still being implemented 
with fidelity after contextualization)? 

e How satisfied are program staff, parents, and other stakeholders with the process of culturally 
adapting the Incredible Years program? 

e What are perceived to be the greatest successes and challenges of the contextualization process? 


Incredible Years staff collected all data, including family entry forms, assessment results, parent 
satisfaction surveys, and other pertinent data. The Incredible Years facilitators administered two 
instruments to the parents at the beginning of program participation and again at the end to measure 
pre/post changes in the participating parents’ stress and children’s social-emotional development: the 
Parenting Stress Index-Short Form (PSI-SF) and the Strengths and Difficulties Questionnaire (SDQ). 
Parent participants also completed a survey at the end of the program to gauge their satisfaction with 
Incredible Years. This information was shared with CAREI. CAREI evaluators also conducted semi- 
structured interviews with key staff members to obtain information about the cultural contextualization, 
fidelity of implementation, and general perceptions about what worked or not throughout the grant period. 
Information from meetings between evaluation staff and the grantees was also included here. 


Impact Evaluation Findings. Analyses were restricted to children and parents who participated in a 
culturally contextualized Incredible Years program (African American or Hmong) and who had complete 
data. During the grant period (April 2013-December 2015), 25 Incredible Years parenting groups were 
offered. Of those, 14 were community groups (i.e., not strictly African American or Hmong cultural 
contextualization groups), 4 were African American cultural contextualization groups, and 7 were Hmong 
cultural contextualization groups. The community groups included parents from a variety of cultural 
groups and were not culturally contextualized. For this evaluation study, all analyses were restricted to 
children and parents who participated in a culturally contextualized Incredible Years program and who 
had complete data. For the African American cultural contextualization cohort (the combined information 
from the 4 African American cultural contextualization groups), 13 pre/post scores on the PSI-SF and 8 
pre/post scores on the SDQ were included in the impact evaluation analysis. For the Hmong 
contextualization cohort, 26 pre/post scores on the PSI-SF and 19 pre/post scores on the SDQ were 
included. 


Inferential and descriptive statistics were used to assess changes in parents and children. For both the PSI- 
SF and SDQ, pre/post differences were examined. In addition, the percent of scores falling into different 
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categories from pretest to posttest were examined. To determine whether there were significant changes 
or effects as a result of participating in the Incredible Years culturally contextualized parenting programs, 
pretest scores and posttest scores on the PSI-SF and the SDQ were compared. Paired ¢ tests were 
computed for the Hmong cultural contextualization cohort, and no significant differences were found for 
either instrument on the total scale or any of the subscales. Thus, there is no evidence in these data to 
suggest that there was any effect on the Hmong cohort. Although f tests were not conducted for the 
African American cultural contextualization cohort because the sample size was too small, effect sizes 
were calculated. The effect sizes were large and in the desired direction in three areas — PSI-SF Total 
Stress score, the PSI-SF Difficulty Child subscale, and the SDQ Prosocial subscale — which suggests that 
further examination of the impact of the African American cultural contextualization is warranted. 


The number of changes in classification categories on the PSI-SF and SDQ was also examined (normal or 
clinical for PSI-SF; abnormal, borderline, or normal for SDQ). For all scales, the majority of children and 
parents remained in the same classification category from pretest to posttest. This is not surprising, given 
that no significant differences between pretest and posttest were detected. For one scale (SDQ Prosocial 
subscale for the Hmong cultural contextualization cohort), the number of children who moved into the 
“worsened” category at posttest was larger than the number in the “improved” category. 


Implementation Evaluation Findings. For the impact evaluation, descriptive statistics and qualitative 
analysis were used to describe implementation and participants’ reactions to the program The following 
aspects of implementation were addressed: program dosage, implementation fidelity, information about 
the cultural contextualizations, and key staff members’ perceptions. 


During the subgrant period, 4 African American cultural contextualization groups and 7 Hmong cultural 
contextualization groups were held. The majority of the groups met for 7 weeks. Thirty-six African 
American and 43 Hmong parents participated in the culturally contextualized groups for a total of 79 
parents. In the African American contextualized cohort, there were 34 interactive children and 55 siblings. 
In the Hmong contextualized cohort, there were 40 interactive children and 45 siblings. 


Information about fidelity to the Incredible Years national curriculum was obtained through interviews 
and meetings with Wilder staff members who were responsible for implementing the Incredible Years 
program during the grant period. The evaluator was unable to observe program sessions in person, 
because it was determined that the evaluator’s presence would be intrusive and would inhibit the parents 
from fully participating in the program. Interviewees explained that the cultural contextualizations were 
adopted in such a way that the key elements of the standard Incredible Years model were preserved. 


The primary cultural contextualization occurred during the first week of programming. Although 
Incredible Years was implemented as usual using the standard Incredible Years lesson plans and 
activities, facilitators allowed time to incorporate cultural conversations into each session. No 
contextualization manual or curriculum was developed and no fidelity checks of the cultural 
contextualization occurred. However, interviewees noted that fidelity to the Incredible Years curriculum 
and programming was maintained using a variety of methods (e.g., completion of weekly facilitator 
checklists, observations by program coordinator, and so forth). Parents also completed a satisfaction 
questionnaire at the end of programming; in general, most parents were satisfied with the program. 


Conclusions and Recommendations. Although the subgrantee struggled with recruiting participants, 
during the grant period, they facilitated 25 groups that served 557 people — 202 parents and 355 children. 
This shows that they were able to deliver the programming to many parents and children. However, in 
order to increase participation numbers, the programming was offered to community groups rather than 
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strictly African American or Hmong contextualization groups. While this resulted in a larger number of 
parents and children receiving the Incredible Years programming, it did not provide large sample sizes for 
testing the hypothesis that culturally contextualized versions of the Incredible Years curriculum would 
improve outcomes for parents and children in the two cultural groups targeted for this intervention. 


Several limitations in the dataset hindered our ability to further examine the data. Recommendations for 
future data collection include: 
e Track daily attendance data; 
e =Track all groups in which a parent participates (dates, age groups, interactive child for each 
group); 
e Attach parent data to their interactive children’s data; 
e Enter the date an assessment was administered rather than the date it was entered into the 
database; and 
e Enter item level information as well as scores for each assessment to allow the sample to be 
checked against the normative sample for each instrument. 


Evaluating implementation was also challenging because evaluators relied on information provided by 
key staff members through meetings, documents, and interviews. Recommendations around 
implementation include: 
e Create a manual for how to culturally contextualize the Incredible Years classroom while 
maintaining fidelity to the national curriculum; 
e Clearly demonstrate how training on cultural contextualization occurs during the three-day 
Incredible Years facilitator training; and 
e Adapt the Incredible Years materials to incorporate aspects of the cultural contextualization: 
o Add items to the observer checklists that include ways in which facilitators add in 
cultural contextualization to the daily lesson; 
o Create a rubric for the program manager to use when observing Incredible Years 
classrooms that includes cultural contextualization pieces; 
o Add items to the parent weekly surveys and to the parent satisfaction survey that 
expressly ask parents whether they felt that their culture had been taken into account or 
that the terminology was understandable. 


The purpose of the subgrant was to test whether an evidence-based program that did not necessarily have 
explicit cultural aspects could be modified to include more culturally appropriate pieces (e.g., 
terminology, video vignettes, examples) and then show the same or better outcomes for the participants. 
This question fits nicely with the theory of change put forth by the SPPN, which posits that “...culture is 
the missing piece of the puzzle in the education process.” The purpose of this grant was to validate the 
theory; however, because of the way it unfolded, the number of people participating in the cultural 
contextualization groups was lower than hoped. Project staff felt that the majority of the subgrant period 
was then spent on “getting the numbers,” rather than focusing on further refining the Hmong cultural 
contextualization and developing the African American version. However, in the process, Incredible 
Years program staff members reported that they learned several lessons: (1) adapting an evidence-based 
program is time-consuming work; (2) it takes time to build trust and rapport in communities, especially 
when you want to offer something new and test it at the same time; (3) recruitment strategies need to also 
be culturally contextualized; and (4) it may be best to partner with organizations that already serve the 
cultural groups your service is targeting, but do not currently offer the programming you provide. The test 
will be how Wilder and SPPN use the lessons they learned from participating in this project to continue to 
infuse culture into their programming. 
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AN EVALUATION OF THE CULTURAL 
CONTEXTUALIZATION OF THE INCREDIBLE YEARS 
PARENTING PROGRAM FOR AFRICAN AMERICAN AND 
HMONG CULTURAL GROUPS 


INTRODUCTION 


This report is based upon work supported by the Social Innovation Fund (SIF), a key White House 
initiative and program of the Corporation for National and Community Service (CNCS). The Social 
Innovation Fund combines public and private resources to grow the impact of innovative, community- 
based solutions that have compelling evidence of improving the lives of people in low-income 
communities throughout the United States. ! 


Greater Twin Cities United Way received funds from the SIF to serve as a grantmaking intermediary in 
Minneapolis and Saint Paul. As part of a competitive selection process, the Amherst H. Wilder 
Foundation (Wilder) was selected as the fiscal agent for the Saint Paul Promise Neighborhood’s (SPPN) 
subgrant. The Center for Applied Research and Educational Improvement (CARED) was awarded the 
contract to evaluate several of the seven nonprofit subgrantee programs that received SIF funding from 
the United Way. 


Through the SIF award, Wilder and SPPN intended to deliver culturally contextualized versions of the 
Incredible Years program to African American and Hmong parent groups in the SPPN. This evaluation 
report will provide information about the cultural contextualization process and report on the 
effectiveness of the culturally-contextualized Incredible Years parent programs, specifically for African 
American and Hmong parents and children. The subgrant period reported here ran from April 15, 2013 — 
December 31, 2015, which includes Year 1 (April 2013-April 2014), Year 2 (April 2014-April 2015), and 
a Year 2 no-cost extension (April 2015-December 2015). 


This report will provide background on the issue that was addressed with the proposed program model — 
the cultural contextualization of the Incredible Years parenting program for African American and 
Hmong parent groups. The program theory, logic model, and outcomes of interest are also described 
because they helped to shape the implementation and impact evaluation questions and study design. 
Results for the implementation and impact evaluation are described followed by a conclusions section that 
summarizes the findings and provides recommendations. 


Problem Definition 


The Saint Paul Promise Neighborhood (SPPN) is part of the U.S. Department of Education’s Promise 
Neighborhood network whose vision “...is that all children and youth growing up in Promise 
Neighborhoods have access to great schools and strong systems of family and community support that 
will prepare them to attain an excellent education and successfully transition to college and a career.” 
The SPPN, located in the Frogtown and Summit-University neighborhoods of St. Paul, is host to a mix of 
many cultures, with the largest proportion of the neighborhood being from the African American and 


1 For more information, see: nationalservice.gov/Innovation. 
? For more information, see: http://www2.ed.gov/programs/promisenei ghborhoods/index.html. 
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Hmong communities.* Within the SPPN, there are glaring racial disparities in early childhood education 
participation. The 2011 SPPN Community Assessment (Karcher-Ramos, MartinRogers, Mueller, & 
Mattessich, 2011) found that only 5% of Hmong and 53% of African American children age birth through 
five living in the SPPN participated in formal early childhood learning programs. The SPPN Early 
Childhood Solution Action Group realized that low participation in early learning settings on the part of 
Hmong families is related to unfamiliarity with, and lack of trust of, non-school based centers, coupled 
with the cultural child rearing practices exhibited in mainstream early learning centers that may not be of 
interest to many Hmong parents. Among African American families, major barriers to participation in 
early learning settings include cost and transportation, as well as other factors. The SPPN’s Social 
Innovation Fund theory of change hypothesizes that “cultural engagement is a critical piece of the 
education puzzle” and states that “all students and families have access to educational opportunities that 
honor and fully integrate their cultural practices, values, communication preferences, and learning styles” 
(Saint Paul Promise Neighborhood, 2015). The belief is that cultural engagement will, in part, increase 
participation in programs because the curriculum is culturally-specific and culturally-relevant. 


For these reasons, the SPPN and Wilder partnered to deliver culturally contextualized curricula of the 
Incredible Years parenting program for these two cultural communities.* With the Social Innovation Fund 
subgrant, SPPN and Wilder proposed to develop an African American contextualization, to refine a 
Hmong contextualization, and to measure the effectiveness of the contextualizations on parenting skills 
and children’s social-emotional development. 


Program Description 


Incredible Years is an evidence-based program that promotes emotional and social competence in young 
children and helps prevent aggression and emotional problems. Incredible Years parenting programs are 
offered to parents of babies (0-12 months), toddlers (1-3 years), preschoolers (3-6 years), and school age 
children (6-12 years). Incredible Years emphasizes a strong parent-child relationship and enhancement of 
parenting skills. Groups typically meet for 12-20 weekly 2-hour sessions. “Group sessions focus on: 
strengthening parent-child interactions, nurturing relationships, reducing harsh discipline, and fostering 
parents’ ability to promote children’s social, emotional, and language development” (Retrieved from: 
http://incredibleyears.com/programs/parent/). In addition, Incredible Years offers the Attentive Parenting 
program, which is a “6-8 session group-based ‘universal’ parenting program” that promotes “children’s 
emotional regulation, social competence, problem solving, reading and school readiness.”° Educating 
parents about child development and parent-child interactions such as praising positive behaviors and 
coaching children through appropriate verbal expression of emotions can have a beneficial impact on 
social-emotional competence and, ultimately, on school readiness (Gray, Heberle, & Carter, 2012).° 


3 For more information on the Saint Paul Promise Neighborhood (SPPN), see: https://www.wilder.org/Community- 
Leadership/Saint-Paul-Promise-Neighborhood/Pages/default.aspx. 

4 For more information on the Incredible Years parenting program, see: http://incredibleyears.com/programs/parent/. 
Additional information can be found at the What Works Clearinghouse: 
http://ies.ed.gov/ncee/wwe/interventionreport.aspx?sid=590. 

> For information on the Incredible Years Attentive Parenting program, see: 
http://incredibleyears.com/programs/parent/attentive-curriculum/. 

® There is a strong link between young children's social-emotional competence and their chances of early school 
success (Raver, 2002). Social-emotional development is also deeply rooted in culturally regulated child-rearing 
practices (Rubin, 1998). Studies have demonstrated that social-emotional knowledge has a critical role in improving 
children's academic performance and life-long learning (Zins, Bloodworth, Weissberg, & Walberg, 2004). Children 
who are aware of their emotions and have good planning skills by the time they enter school also have a lower risk 
for problems of aggression and anxiety disorders (Greenberg, Kusch, & Mihalic, 1998). Special programs that 
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Prior to applying for SIF funding, Wilder worked with the University of Minnesota to adapt the Incredible 
Years model to be culturally-specific and relevant for Hmong parents and children. Examples of the ways 
in which the curriculum was culturally contextualized for Hmong families include the use of a Hmong- 
only group format and changes to the symbols used in the curriculum (e.g., participants found the use of a 
house image more culturally meaningful than the image of a pyramid to represent approaches to building 
parent/child relationships). (Appendix A provides background information on the development of the 
Hmong cultural contextualization.) As part of the SIF funding, SPPN and Wilder wanted to further test 
this cultural adaptation for Hmong families, as well as replicate the cultural adaptation for African 
American parents and children. 


In the subgrant proposal, Wilder proposed to deliver culturally-contextualized curricula of Incredible 
Years designed for Hmong parents and caregivers of children in four different age groups (0-12 months, 
1-3 years, 2-4 years, and 5-6 years) and to partner with the University of Minnesota to offer newly 
culturally-contextualized curricula for African American parents and caregivers of children in the same 
four age groups. (Appendix B offers a descriptive overview of the four Incredible Years program types 
offered by Wilder: the Parents and Babies Program, the Parents and Toddlers Program, the Parents and 
Toddlers Attentive Parenting Program, and the Early School Age (Preschool) Attentive Parenting 
Program.) Each participating parent(s) is asked to select a focal or “interactive” child for the program. 
The interactive child joins the parent at the end of each session to work on techniques that were taught to 
the parent. The program provides childcare for the interactive child and siblings during each 2-hour 
session. Originally, Wilder intended to recruit parents and children from SPPN-sponsored sites and 
programs and from other early learning settings affiliated with the SPPN Early Learning Network; 
however, offerings were expanded outside this network through the subgrant period. In addition to the 
cultural contextualization of the programming, Wilder and SPPN indicated that they would offer 
transportation assistance, childcare, a shared family meal, and a weekly incentive to help families 
overcome barriers to participation and encourage their ongoing participation. 


Program Theory, Logic Model, and Outcomes of Interest 


The logic model for the Incredible Years cultural contextualizations (see Appendix C) is a modification of 
the logic model for the traditional Incredible Years Parent Program developed by Carolyn Webster- 
Stratton at the University of Washington. The model presents program components and goals, program 
modalities, targeted risk and protective factors, proximate (short-term) outcomes, and distal (long-term) 
outcomes. 


The components of the program are the Incredible Years BASIC parent program and developmentally 
specific content for three age groups: babies, toddlers, and preschool children. The goal of the program is 
to enhance positive parenting interactions, coaching, attachment with children, and proactive discipline. 


The program modalities are the specific strategies, methods, and techniques used to accomplish the 
program goals. These modalities include goal setting and self-monitoring; video monitoring; role play and 
behavioral practice; group support, discussion, and problem solving; cognitive emotional-regulation 
training; take-home practice and reading; and family meal and child care. 


promote social-emotional learning reduce aggression and increase prosocial behavior (Schonert-Reichl, Smith, 
Zaidman-Zait, & Hertzman, 2012; Weissberg, Kumpfer, & Seligman, 2003). 
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The program targets risk factors and protective factors. The risk factors are related to the parent, child, 
and family. The risk factors include parent personal/interpersonal problems; ineffective parenting skills; 
low monitoring; numerous stressors (e.g., challenging life events); low parent involvement with schools 
and teachers; and child biological factors (e.g., developmental delays). The program also targets 
protective factors related to the parents. These protective factors include positive, nurturing, 
developmentally appropriate parenting skills; parent support for the child’s social, emotional, and 
academic development; parent support networks; parent partnerships with teachers and schools; and 
effective communication skills, anger management, and problem solving. 


The short-term outcomes are targeted outcomes that the program has been shown in research to impact 
immediately and 1 to 3 years following program completion. Short-term outcomes are targeted in three 
areas: improved parenting and interpersonal skills, improved child behavior at home, and improved child 
behavior at school. The long-term outcomes for youth, designed to impact at long-term follow up during 
adolescence, are related to decreased antisocial behavior, less depression, and less likelihood of dropping 
out of school. 


Research on the impact of parenting education programs has shown that such programs can significantly 
improve participants’ knowledge of effective parenting strategies (Winter, Morawska, & Sanders, 2012); 
the quality of parent-child interactions; the occurrence of a child’s positive behaviors in non-school 
settings; and parental depression (e.g., Niccols, 2009). Similarly, an investigation of the effects of the 
Incredible Years parent training program on outcomes for children aged 3 to 7 years and their parents 
reported significant improvements in children’s problem behaviors and social competence as well as 
increases in parental well-being and competence (McGilloway, Ni Mhaille, Bywater, Furlong, Leckey, 
Kelly, et al., 2012). In addition, research focused on school-related outcomes has found that a parenting 
intervention emphasizing positive behavior support, monitoring and limit setting, and enhanced 
relationship quality carried out with parents of children 2 to 3 years of age resulted in higher academic 
performance at ages 5 and 7.5 years when compared to the performance of children whose parents did not 
participate in the intervention (Brenna, Shelleby, Shaw, Gardner, Dishion, & Wilson, 2013). 


METHODS 
Purpose 


The evaluation study was designed to measure the effect of two Incredible Years cultural 
contextualizations on outcomes of African American and Hmong parents and interactive children. In 
addition, the evaluation was designed to describe the implementation of the contextualized versions of the 
Incredible Years parenting program. 


Evaluation Questions 


IMPACT EVALUATION QUESTIONS 


Prior to receiving SIF funding, the Hmong culturally contextualized program had been developed and 
offered once. Wilder also proposed, as part of this grant, to develop an African American cultural 
modification of the Incredible Years model. Because the Hmong contextualization was early in the testing 
stage and the African American contextualization had not yet been developed at the start of the grant 
period, the evaluation team determined that it was appropriate and feasible to obtain a preliminary level of 
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evidence for each of these contextualizations by Year 2, using the data from Years 1| and 2. For the impact 
evaluation, confirmatory and exploratory questions were proposed.’ 


The confirmatory questions for Years 1 and 2 are: 

e Do parents who participate in the culturally contextualized versions of Incredible Years show a 
reduction in parental stress, as measured by the Parenting Stress Index-Short Form (PSI-SF; 
Abidin, 1995)? 

e Do parents who participate in the culturally contextualized versions of Incredible Years exhibit an 
increase in effective parenting skills, as measured by the PSI-SF? 

e Dochildren participating in the culturally contextualized versions of Incredible Years show 
improvements in social-emotional behaviors, such as emotional problems, conduct problems, 
hyperactivity-inattention, peer problems, and pro-social behavior as measured by the Strengths 
and Difficulties Questionnaire (SDQ; Goodman, 1997)’? 


IMPLEMENTATION EVALUATION QUESTIONS 


In this early stage of developing cultural contextualizations for the African American and Hmong 
communities of the standard Incredible Years model, the evaluation team also proposed studying program 
implementation. The implementation evaluation was designed to examine whether the project was 
implemented as proposed. The following questions were addressed by the implementation evaluation: 

e Are parents and children receiving the designated amount of treatment programming? 

e Is the Incredible Years Program being implemented as intended? 

e Are the culturally contextualized versions of the Incredible Years model incorporating all of the 
key elements of the standard Incredible Years model (i.e., is the program still being implemented 
with fidelity after contextualization)? 

e How satisfied are program staff, parents, and other stakeholders with the process of culturally 
adapting the Incredible Years program? 

e@ What are perceived to be the greatest successes and challenges of the contextualization process? 


Data Collection 


Incredible Years staff collected all data, including family entry forms, assessment results, parent 
satisfaction surveys, and other pertinent data. Most of that information was then entered by Incredible 


7 Two exploratory questions were posed in the approved SIF evaluation plan, but they will not be addressed here. 
The first question, “Is the abbreviated series (6 or 7 weeks) of the culturally contextualized version of Incredible 
Years as effective as the longer series (10-14 weeks) as indicated by results on the outcome measures utilized to 
assess program impact?” could not be addressed because the sample sizes were not large enough to study dosage 
effects. The second question, “Are the instruments that have been utilized to assess outcomes of the traditional 
Incredible Years program (PSI-SF, SDQ, ASQ, and ASQ:SE) valid for assessing outcomes of the Hmong and 
African American culturally contextualized versions of Incredible Years?” could not be addressed because 
Incredible Years’ childcare workers did not complete the assessments or comparison of ratings as described in the 
approved SIF evaluation plan; thus, information was not obtained that would allow for the planned instrument 
validation study. 

8 In the final approved evaluation plan, it was also listed that the Ages and Stages Questionnaire (ASQ; Squires, 
Twombly, Bricker & Potter, 2009) and the Ages and Stages Questionnaire: Social-Emotional (ASQ:SE; Squires, 
Bricker, & Twombly, 2002) would be administered for younger children; however, only a small number were 
administered and there were administrative errors during administration so information from the ASQ and ASQ:SE 
was not available. 
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Years staff into CareLogic, the data system used by Wilder’s Child Development Center, and shared with 
the SPPN. Parents of Incredible Years participants signed a consent form that allowed Wilder to enter and 
store information in the SPPN database and gave permission for SPPN to share data with CAREI 


The Incredible Years facilitators were asked to administer two instruments to the parents at the beginning 
of program participation and again at the end to measure pre/post changes in the participating parents’ 
stress and childrens’ social-emotional development. These assessments include: the Parenting Stress 
Index-Short Form (PSI-SF; Abidin, 1995) and the Strengths and Difficulties Questionnaire (SDQ; 
Goodman, 1997). The PSI-SF is completed by parents and measures the level of parental distress, positive 
parent-child interactions, and parent’s perceptions of the child’s behavior. The SDQ is also completed by 
parents and focuses on the child they have selected for class interaction time (the “interactive” child). The 
SDQ provides a measure of the child related to emotional problems, conduct problems, hyperactivity, 
peer relationship problems, and pro-social behavior. In addition to completing the pre/post parental stress 
and social-emotional development instruments, parent participants completed a survey at the end of the 
program to gauge their satisfaction with Incredible Years. 


INSTRUMENTS 


PSI-SF instrument. The PSI-SF has three subscales (Parental Distress, Parent-Child Dysfunctional 
Interaction, and Difficult Child) consisting of 12 items each. Parents respond using a 5-point scale 
(strongly agree, agree, not sure, disagree, strongly disagree) to indicate how well each item represents 
their opinions.’ The Parental Distress subscale score indicates degree of distress arising from personal 
factors such as depression and stressful demands related to life role restrictions (e.g., “J feel trapped by 
my responsibilities as a parent”). The Parent-Child Dysfunctional Interaction subscale yields a score that 
indicates the level of dissatisfaction with the parents’ exchanges with their child and the extent to which 
parents feel their child does not meet expectations (e.g., “When I do things for my child, I get the feeling 
that my efforts are not appreciated very much’’). The Difficult Child subscale provides an indication of 
parents’ perceptions of their child’s self-regulatory abilities (e.g., “My child reacts very strongly when 
something happens that my child doesn’t like’’). In addition, a Total Stress score indicates the overall 
stress level experienced by the parents. 


SDQ instrument. The SDQ consists of 25 items that are divided among five scales of five items each. 
Subscale scores are generated for the five subscales: Emotional Problems, Conduct Problems, 
Hyperactivity, Peer Problems, and Prosocial Behavior. Example statements for the Emotional Problems 
subscale include, “Often complains of headaches, stomach-aches or sickness” and “Many worries or 
often seems worried.” Example statements for the Conduct Problems subscale include, “Often fights with 
other children or bullies them” and “Generally well behaved, usually does what adults request.” 
Example statements for the Hyperactivity subscale include, “Restless, overactive, cannot stay still for 
long” and “Good attention span, sees chores or homework through to the end.” Example statements for 
the Peer Problems subscale include, “Picked on or bullied by other children,” and “Has at least one good 
friend.” Example statements for the Prosocial Behavior subscale include, “Considerate of other people’s 
feelings” and “Kind to younger children.” A Total Difficulties score is also provided. Respondents rate 
the extent to which statements reflect their child’s behavior on a 3-point Likert-type scale (not true, 
somewhat true, certainly true).'° 


° Parents are also measured on “Defensive Responding,” which is a validity subscale that indicates whether a parent 
is answering in a way that would make him/her look more favorable. 
'0 Ror more information, see www.sdqinfo.org. 
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Parent Satisfaction Survey."! Parents were administered the Incredible Years Parent Program Satisfaction 
Questionnaire (Attentive Parenting Program version) at the end of programming. This measure of parent 
satisfaction provides information for the implementation evaluation. Participating parents were asked 
questions about the overall program (7 questions), the teaching format (7 questions), the specific 
parenting techniques (7 questions), the parent group leader(s) (4 questions), and the parent group (4 
questions). Each question had one of several 7-point response scales; for example, parents were asked to 
rate the following statement, “My child’s social and emotional skills are” using a scale of considerably 
worse, worse, slightly worse, the same, slightly improved, improved, or greatly improved. The survey 
also includes three open-ended items about recommendations for improvement, need for further parenting 
assistance, and benefits of the program. 


Interviews. Semi-structured interview protocols were developed to ask program staff about the cultural 
contextualization, fidelity of implementation, recruitment, and general perceptions about what worked or 
not throughout the grant period. The CAREI evaluator conducted interviews with the Director of the Saint 
Paul Promise Neighborhood (SPPN); the Director of Wilder’s Child Development Center; the Incredible 
Years Coordinator; a Parent Education Recruiter and Community Outreach Coordinator and Incredible 
Years facilitator; two Incredible Years facilitators; and a University of Minnesota professor. In addition, 
several meetings were held with these key staff members, as well as two data specialists from Wilder, 
from June 2015 through December 2015. Information from these meetings was also included here. 


Study Design 


Because some parents and children left the program before all data were collected, missing data were 
present in the dataset. For this evaluation study, all analyses were restricted to children and parents who 
participated in a culturally contextualized Incredible Years program and who had complete data. 
Inferential and descriptive statistics were used to assess changes in social-emotional development, child 
behavior, parenting skills, and parental distress; descriptive statistics and qualitative analysis were used to 
describe implementation and participants’ reactions to the program. 


IMPACT EVALUATION DESIGN 

For both the PSI-SF and SDQ, pre/post differences were examined. In addition, the percent of scores 
falling into different categories from pretest to posttest were examined (see Table 1 for more 
information). Only participants with PSI-SF pretest and posttest data were included in the PSI-SF 
analyses; only those with pre/post data on the SDQ were included in the SDQ analyses. 


Table 1. Impact Evaluation Questions and Information Needed 


Evaluation Question Information Needed 
e Do parents who participate in the culturally Parenting Stress Index-Short Form (PSI-SF) 
contextualized versions of Incredible Years e Pre/Post information completed by parents 
show a reduction in parental stress? tested for significant differences between 
e Do parents who participate in the culturally pretest and posttest on three subscale scores 
contextualized versions of Incredible Years and Total Stress score 


'! The final approved SIF evaluation plan indicated that weekly parent surveys would be administered and analyzed. 
The weekly survey was used for internal purposes only to monitor implementation fidelity. The information 
analyzed here is from the Incredible Years Parent Program Satisfaction Questionnaire (Attentive Parenting Program 
version) administered to parents at the end of the program. 
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Evaluation Question Information Needed 
exhibit an increase in effective parenting e Number in normal and clinical categories at 
skills? pretest and posttest 
e Number of pretest to posttest classification 
category changes 


e Do children participating in the culturally Strengths and Difficulties Questionnaire (SDQ) 
contextualized versions of Incredible Years e Pre/Post information completed by parents 
show improvements in social-emotional tested for significant differences between 
behaviors? pretest and posttest on five subscale scores and 

o SDQ. Will children participating in the Total Difficulties score 
Incredible Years program who are e Number in normal, borderline, or abnormal 
assessed by the SDQ show improvement categories at pretest and posttest 
in emotional symptoms, conduct e Number of pretest to posttest classification 
problems, hyperactivity, peer problems, category changes 


and prosocial behavior? 


For the African American cohort that received the culturally contextualized Incredible Years 
programming, sample sizes on both the PSI-SF and the SDQ were too small to perform a ¢ test analysis to 
test whether the differences between pretest and posttest were statistically significant for the overall scale 
score and subscale scores. However, effect sizes were calculated and are reported here. An appropriate 
effect size measure for the difference between two means is Cohen’s d, which is the mean difference in 
standard deviation units. A conventional interpretation of d is: 

e 0.8 = large effect size (almost an entire standard deviation) 

e 0.5 = moderate effect size 

e 0.2 = small effect size. 


For the Hmong cohort that received the culturally contextualized Incredible Years programming, matched 
pairs ¢ tests were performed to test whether the differences between pretest and posttest were statistically 
significant for the overall scale score and subscale scores on both the PSI-SF and the SDQ. To test the 
robustness of our findings against violations of normality, a series of signed rank tests were also 
performed, and no differences were found between the ¢ test results and the signed rank results (i.e., there 
was 100% agreement between the tests). Therefore, the f test findings will be presented in the results 
section for the Hmong cohort, along with information on effect sizes for the overall scale score and 
subscale scores. 


For both the African American and Hmong cohorts, the number of scores falling into the categories of 
“normal” and “clinical” at each test administration of the PSI-SF as well as the number of pre/post 
category shifts were examined. Similarly for the SDQ, for each cohort (African American and Hmong), 
the number of scores falling into the “abnormal,” “borderline,” or “normal” categories at both the pretest 
and posttest is reported for the Total Difficulties score and each subscale. The number of pre/post 
category shifts is also presented. 


IMPLEMENTATION EVALUATION DESIGN 


The purpose of the implementation evaluation was to verify that the project was implemented as 
proposed. The following aspects of implementation are addressed: program dosage; implementation 
fidelity; information about the cultural contextualizations; and key staff members’ perceptions of what 
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worked well, challenges, and lessons learned during the grant period. Table 2 shows the information 
obtained and reported for each implementation evaluation question. 


Table 2. Implementation Evaluation Questions and Information Needed 


Evaluation Question 


Information Needed 


Are parents and children 
receiving the designated 
amount of treatment 
programming? 


Program dosage information: 

e Number of parents, interactive children, and siblings in the African 
American and Hmong culturally contextualized cohorts 

Age ranges for interactive children 

Number of hours of programming and childcare 

Number and race/ethnicity of facilitators 

Dates of programming 


Is the Incredible Years 
Program being implemented 
as intended? 

Are the culturally 
contextualized versions of the 
Incredible Years model 
incorporating all of the key 
elements of the standard 
Incredible Years model (1.e., 
is the program still being 
implemented with fidelity 
after contextualization)? 


Information on program delivery: 
e Number of instructors who are African American and Hmong 
e Number of instructors who speak the Hmong language 
o Documents provided by Wilder 
Information about implementation fidelity: 
e Interview data that provides information about how fidelity of 
implementation was monitored 
o Interviews with key staff members, including the Director of 
SPPN; the Director of Wilder’s Child Development Center; the 
Incredible Years Coordinator; a Parent Education Recruiter and 
Community Outreach Coordinator and Incredible Years 
facilitator; two Incredible Years facilitators; and a University of 
Minnesota professor 
Information about satisfaction with Incredible Years programming: 
e Incredible Years Parent Program Satisfaction Questionnaire 
(Attentive Parenting Program) results 
o Data collected by, and provided to CAREI, by Wilder 


How satisfied are program 
staff, parents, and other 
stakeholders with the process 
of culturally adapting the 
Incredible Years program? 


Information about cultural contextualization: 

e Interview data that provides information about the cultural 
contextualization that occurred 
o Interviews with key staff members 

e Document analysis and meeting notes 


What are perceived to be the 
greatest successes and 
challenges of the 
contextualization process? 


Feedback from staff members regarding: 

e Perceptions of program implementation (what worked well, 
challenges, lessons learned) and perceptions of impact on families 
o Interviews with key staff members 
o Meeting notes 


Information regarding program dosage was obtained from interviews and meetings with key project staff; 
from documents, provided by project staff, listing group and facilitator information; and from information 
in the CareLogic dataset that Wilder provided to CAREI.'? Information about implementation fidelity and 
the cultural contextualizations was obtained through interviews and meetings with key project staff. 
Another measure of implementation fidelity included here is the degree of satisfaction with the program 


” The following aspects of program dosage that were proposed in the approved SIF evaluation plan will not be 
reported here due to lack of available data: attendance rates of parents and children, session locations, session times. 
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by participants. This was measured by the parent satisfaction questionnaire completed by parents at the 
end of the program.'? Finally, interviews and meetings with key project staff also provided information 
about successes, challenges, and lessons learned by the staff throughout the grant period. 


For the parent satisfaction survey, the percent of respondents selecting each response option was 
calculated for each survey question. In addition, the information provided in the open-ended questions 
was summarized. The qualitative data from the interviews was analyzed to identify common issues and 
themes and to provide additional context and information about the cultural contextualization process and 
implementation, as well as strengths and challenges experienced during the grant period. 


RESULTS 


Summary Information 


Recall that Incredible Years is a program offered to parents to strengthen parenting skills and to help 
parents foster their children’s social-emotional skills. Each participating parent(s) is also asked to select a 
focal or “interactive” child for the program. The interactive child joins the parent at the end of each 
session to work on techniques that were taught to the parent. The program provides childcare for the 
interactive child and siblings during the session. 


During the subgrant period (April 2013-December 2015), 25 groups were offered. Of those, 14 were 
community groups" (offered to a mix of people), 4 were African American cultural contextualization 
groups, and 7 were Hmong cultural contextualization groups. The number of sessions varied across the 25 
groups: 

The majority of the groups met for 7 sessions (14 groups), 

3 groups met for 6 sessions, 

3 met for 8 sessions, 

1 met for 9 sessions, 

1 met for 10 sessions, and 

3 met for 12 sessions. 


Each group met once weekly for a 2-hour session. Nineteen facilitators ran the various groups — 8 were 
African American, 6 were Caucasian, 4 were Hmong, and 1 was Korean. In general, African American 
and Hmong facilitators worked with the African American and Hmong contextualization groups, 
respectively.'> In all, 557 people were served — 202 parents and 355 children. Because the purpose of this 
evaluation was to study the cultural contextualizations of the Incredible Years parenting program 
specifically offered to African American or Hmong parents and children, findings from only the 
contextualization groups will be reported here. For analysis purposes, information from the four African 
American cultural contextualization groups were combined and are called the African American cohort; 


'3 The following information about implementation and the cultural contextualizations, proposed in the final 
approved SIF evaluation plan, was not provided for analysis: lesson plans, teaching materials, and handouts 
specifically related to the cultural contextualization; videotape recordings of group sessions; minutes of instructional 
time in the Hmong language (which could not be determined because facilitators spoke using a mix of English and 
Hmong). 

“4 The community groups included parents from a variety of cultural groups and were not culturally contextualized. 
'S For one of the Hmong cultural contextualization groups, a Hmong and Caucasian facilitator worked together. 
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information from the seven Hmong cultural contextualization groups were combined and are called the 


Hmong cohort.!° 


SAMPLE 


Impact Evaluation Findings 


For the impact evaluation, the following inclusion criteria were applied to the dataset provided to CAREI 
by Wilder — the parent or child had to be in either an African American or Hmong group (if the group 
name included “community” then the participant was excluded) and only interactive children were 
included and not siblings (data were not collected on siblings). In addition, for the inferential statistics, a 
parent’s data was included only if there was both a pretest and posttest. For cases in which a child had 
more than one pretest or posttest score on the SDQ or a parent had more than one pretest or posttest score 
on the PSI-SF, the data were excluded. Given those criteria, a total of 79 parents and 74 interactive 
children were included in the final dataset (see Table 3 for breakdowns by cultural contextualization 
cohort). Of the 79 parents, 39 parents had pretest and posttest data that could be included in the analysis 
for the PSI-SF (13 were African American and 26 were Hmong) and 27 parents had pretest and posttest 
data that could be included in the SDQ analysis (8 were African American and 19 were Hmong)."” 


Table 3. Number of Cases Included in Impact Analysis 


Cultural Number of Number Included | Number Included | Number of | Number 
Contextualization Parents in PSI-SF Impact | inSDQImpact | Interactive of 
Cohort Analysis Analysis Children* | Siblings** 
African American 36 13 8 34 55 
Hmong 43 26 19 40 45 
Total 79 39 27 74 100 


* Interactive children participate in the parent-child interaction sessions. 
**Siblings are for reference only; they were not administered any assessments or included in the study. 


Appendices D and E provide the pretest and posttest means, ¢ tests (Hmong cohort only), Cohen’s d,'* and 
changes in classification categories for the Total Stress score as well as for the three subscales (Parental 


Distress, Parental Child Dysfunctional Interaction, and Difficult Child) on the PSI-SF. The same 


information is provided for the SDQ Total Difficulties Score and for the five SDQ subscales (Emotional, 
Conduct, Hyperactivity, Peer Problems, and Prosocial). For both the PSI-SF and the SDQ, lower scores 
are better than higher scores (so a lower posttest score is indicative of an improvement on the measure) 
for all of the scale and subscale scores except for the SDQ Prosocial subscale (where a higher posttest 
score indicates an improvement). Appendix D shows the results for the African American 
contextualization cohort (see Figures D1-D10), and Appendix E shows the results for the Hmong 
contextualization cohort (see Figures E1-E10). 


‘6 Cohort refers to the total set of groups that received the cultural contextualization (e.g., the African American 
cohort refers to the four African American culturally contextualized groups). Group refers to the offering of the 
Incredible Years parenting program that occurred with different groups of parents and children over the subgrant 
period (e.g., there were seven Hmong culturally contextualized groups); group is a subset of cohort. Finally, session 
refers to one of several meeting times when a group met (ranging from 6 sessions to 12 sessions). 
'7 Dosage ranged from 6 sessions to 12 sessions for the two cultural contextualization cohorts. Although examining 
dosage effects would be interesting, the sample sizes were too small to study the impact of dosage. 
'8 Cohen’s d was calculated as the absolute value of the mean of the differences between the pretest and posttest 
divided by the pooled standard deviation of these differences. Thus, when interpreting Cohen’s d, it is important to 


examine the pretest and posttest means to determine whether or not the effect was in the desired direction. 
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TESTS OF PRE/POST DIFFERENCES 


The purpose of the impact evaluation was to determine whether there were significant changes or effects, 
as determined by scores on the PSI-SF and the SDQ, as a result of participating in the Incredible Years 
culturally contextualized parenting programs for African American or Hmong parents and children. To do 
this, pretest scores and posttest scores on the PSI-SF and the SDQ were compared. Because of small 
sample sizes, t tests could not be performed for the African American contextualization cohort. Matched 
pairs ¢ tests were performed on the Hmong cohort, but there were no significant differences from pretest 
to posttest. Effect sizes were computed for both cohorts, and Table 4 shows the summary effect size 
findings. Note that two large effect sizes were found for the African American contextualization cohort, 
suggesting that there was a large difference from pretest to posttest on the PSI-SF Difficult Child subcale 
(Cohen’s d= 0.95) and on the SDQ Prosocial subscale (d = 1.02). On the PSI-SF Difficult Child subscale, 
the posttest mean (20.8; SD = 6.2) was lower than the pretest mean (27.0; SD = 9.1), which shows an 
improvement in this area (see Figure D4, Appendix D). For the SDQ Prosocial subscale, where a higher 
score suggests an improvement, the posttest mean (8.6; SD = 1.3) was higher than the pretest mean (7.0; 
SD = 2.1) (see Figure D10). The effect size of 0.74 on the PSI-SF Total Stress score for the African 
American contextualization cohort is also worth examining further. Again, the posttest mean (62.0; SD = 
16.4) was lower than the pretest mean (74.5; SD = 21.6), suggesting an improvement (see Figure D1). 
These effect size findings suggest that with more power (larger sample sizes), a significant difference (or 
effect) could be detected. 


Table 4. Summary of Effect Sizes on PSI-SF and SDQ for African American and Hmong Cultural 
Contextualization Cohorts 


Cohort and Instrument Scale Effect Size: Cohen’s d 
/ Subscale Small (.2 - <.5) | Moderate (.5 - <.8) | Large (.8 and greater) 

African American PSI-SF 

e Total Stress 0.74 

e Parental Distress 0.56 

e Parent-Child 0.26 

Dysfunctional Interaction 

e Difficult Child 0.95 
Hmong PSI-SF 

e Total Stress 0.36 

e Parental Distress 0.22 

e Parent-Child 0.42 

Dysfunctional Interaction 

e Difficult Child 0.29 
African American SDQ 

e Total Difficulties Score 0.39 

e Emotional Problems 0.25 

e Conduct Problems 0.41 

e Hyperactivity 0.42 

e Peer Problems 0.24 

e Prosocial 1.02 
Hmong SDQ 

e Total Difficulties Score 0.57 
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Cohort and Instrument Scale Effect Size: Cohen’s d 
/ Subscale Small (.2 - <.5) | Moderate (.5 - <.8) | Large (.8 and greater) 
e Emotional Problems 0.39 
e Conduct Problems 0.41 
e Hyperactivity 0.27 
e Peer Problems 0.34 
e Prosocial 0.34* 


* This effect represented a negative effect (see Figure E10 on SDQ Prosocial subscale for the Hmong cohort). 
CHANGES IN CLASSIFICATION CATEGORIES 


The number of changes in classification categories on the PSI-SF and SDQ was also examined. Table 5 
shows the number of parents and children whose classifications (normal or clinical for PSI-SF; abnormal, 
borderline, or normal for SDQ) improved, stayed the same, or worsened from the pretest to the posttest. 
For all scales, the majority of parents and children remained in the same classification category from 
pretest to posttest. This is not surprising, given that no significant differences between pretest and posttest 
were detected. There is only one instance in which the number of children who moved into the 
“worsened” category at posttest was larger than the number in the “improved” category (for the SDQ 
Prosocial subscale for the Hmong cultural contextualization cohort, 5 children had scores in the 
“worsened” category and 2 had scores in the “improved” category; see Figure E10, Appendix E). 


For the African American contextualization cohort, across all scales on the PSI-SF, 17% of the 
classification categories improved from pretest to posttest, 77% stayed the same, and 4% worsened. On 
the SDQ, 29% of the classification categories improved, 63% stayed the same, and 8% worsened. 


For the Hmong contextualization cohort, across all scales on the PSI-SF, 20% of the classification 
categories improved from pretest to posttest, 73% stayed the same, and 7% worsened. On the SDQ, 15% 
of the childrens’ classification categories improved, 74% stayed the same, and 11% worsened. 


Table 5. Summary of Classification Changes: Number of Children Whose Classification Category 
Improved, Stayed the Same, or Worsened 


Cohort and Instrument Scale Classification Change from Pretest to Posttest 
/ Subscale Improved Stayed the Same Worsened 
African American PSI-SF (n=13) 
e Total Stress 2 9 1 
e Parental Distress 3 10 0 
° Parent-Child 5) 10 1 
Dysfunctional Interaction 
e Difficult Child 2 11 0 
Hmong PSI-SF (n=26) 
e Total Stress 6 18 2 
e Parental Distress b) 18 3 
e Parent-Child 
Dysfunctional Interaction : a . 
e Difficult Child 4 22 0 
African American SDQ (n=8) 
e Total Difficulties Score 2 5 1 
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Cohort and Instrument Scale Classification Change from Pretest to Posttest 
/ Subscale Improved Stayed the Same Worsened 

e Emotional Problems 2 6 0 

e Conduct Problems 2 5 1 

e Hyperactivity 2 6 0 

e Peer Problems 3 3 2 

e Prosocial 3 5 0 
Hmong SDQ (n=19) 

e Total Difficulties Score 3 14 2 

e Emotional Problems 2 16 1 

e Conduct Problems 4 13 2 

e Hyperactivity 2 16 1 

e Peer Problems 4 13 2 

e Prosocial 2 12 5 


Implementation Evaluation Findings 


The implementation evaluation was designed to provide information that the project was implemented as 
proposed. The following sections provide information about implementation: program dosage, 
implementation fidelity, cultural contextualizations, and key staff members’ perspectives. 


PROGRAM DOSAGE 


Because the purpose of this evaluation was to study the cultural contextualizations of the Incredible Years 
parenting program specifically offered to parents and children from African American or Hmong cultural 
communities, program dosage information will be reported only for the contextualization groups (see 
Table 6'°). This information is important in order to assess whether or not parents and children received 
an appropriate amount of treatment programming. 


From April 2013-December 2015 (the grant period), 4 African American cultural contextualization 
groups and 7 Hmong cultural contextualization groups were held. The majority of the groups met for 7 
weeks/sessions (5 groups); 1 met for 6 weeks; 2 met for 8 weeks; 1 met for 9 weeks; and 2 met for 12 
weeks. Each weekly session was 2 hours long. Four different facilitators worked with the African 
American cultural contextualization groups, all of whom were African American. Five different 
facilitators worked with the Hmong groups — 4 were Hmong and | was Caucasian. Two of the Hmong 
facilitators worked with 3 groups each, 1 worked with 2 groups, and 1 worked with 1 group; the 
Caucasian facilitator worked with 1 Hmong group. Although Incredible Years does not require two 
facilitators, Wilder has found that to be a successful strategy, so six of the eleven groups had two 
facilitators. 


Thirty-six African American and 43 Hmong parents participated in the culturally contextualized groups 
for a total of 79 parents (see Table 3 above). In the African American contextualized cohort (which 


19 The information reported in Table 6 was provided to CAREI by Wilder in a document, rather than being taken 
from the CareLogic dataset. The dates of programming in Table 6 do not always correspond to the begin dates 
provided in the CareLogic dataset for the participants determined to be eligible for inclusion in the impact 
evaluation. 
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consists of those who participated in one of the 4 African American culturally contextualized groups), 
there were 34 interactive children and 55 siblings. In the Hmong contextualized cohort, there were 40 
interactive children and 45 siblings. The ages of the interactive children ranged from 1 to 7, with the 
majority between the ages of 3 and 6. This is to be expected, because Wilder focused on providing 
services to parents and children in this age range. 


Table 6. Program Information for African American and Hmong Cultural Contextualization Groups 


Cultural Number of Hours of Race / Race / 
Contextualization Dates Group Weeks Programming | Ethnicity of | Ethnicity of 
Group Met Group Met | / Child Care” | Facilitator 1 | Facilitator 2 
African American September 24 — ; ; 
November 26, 7 14 hours catican ican 
American American 
2013 
African American January 21 — 2 94 hours African African 
April 22, 2015 American American 
African American September 9 — 4 one African African 
October 28, 2015 American American 
African American | April 14— June 2, 8 ae African None Raed 
2015 American 
Hmong October 8 — 
November 26, e | 14 hours Hmong Hmong 
2013 
pmone ADAM aney 2°: 7 14 hours Hmong None listed 
2014 
Ene oan 2 a 5 7 14 hours Hmong None listed 
Hmone ae. an 5 12 24 hours Hmong None listed 
none Apo ae BNE, 6 12 hours Hmong Caucasian 
Hmong October 1 — 
December 10, 9 18 hours Hmong Hmong 
2015 
Hmong October 2 — 
December 11, 8 16 hours Hmong None listed 
2015 


IMPLEMENTATION FIDELITY 


An examination of implementation fidelity was important to ensure that the core components of the 
Incredible Years program were retained, even when offering a culturally contextualized version of the 
Incredible Years program. Information about fidelity to the Incredible Years national curriculum was 
obtained through interviews and at meetings with Wilder staff members who were responsible for 
implementing the Incredible Years program during this grant period. The evaluator was unable to observe 
program sessions in person, because it was determined that the evaluator’s presence would be intrusive 


20 Child care was offered for the length of each session; each session was typically two hours long. 
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and would inhibit the parents from fully participating in the program. An alternative was proposed in the 
final approved SEP to videotape sessions, but no videotaping of class sessions was conducted. 


Interviewees explained that the cultural contextualizations were adopted in such a way that the key 
elements of the standard Incredible Years model were preserved. One facilitator noted, “I don't think we 
really changed the curriculum because we still want to stay true to the IY curriculum because ... it's 
considered evidence-based so we still want to stay true to the topics and curriculum, and I think it works 
well when you just allow time to incorporate the conversations of culture into the session.” 


The primary cultural contextualization occurred at the beginning of the program — in the first and/or 
second meeting of the group through the use of a cultural contextualization resource that helps facilitators 
establish rapport and relationships with parents called INTERFACE. INTERFACE stands for 
Incorporating Narratives To Engage and Retain Families through Action, Cultural Connection, and 
Empowerment, and it is currently under development by Dr. Richard M. Lee at the University of 
Minnesota. 


Although Incredible Years was implemented as usual using the standard Incredible Years lesson plans 
and activities, facilitators allowed time to incorporate cultural conversations into each session (see below 
for more information about the cultural contextualization process). It should also be noted that because 
Wilder was having difficulty recruiting families for longer sessions, a decision was made during the grant 
period to offer only the Attentive Parenting Program to parents with children aged 3 to 5 years, which 
reduced the program to 7 weeks.”! This meant that the INTERFACE session was reduced from 2 weeks to 
1 week before Incredible Years programming began. 


No fidelity checks of the cultural contextualization occurred. However, fidelity to the Incredible Years 
curriculum and programming was maintained using the following methods: 
e All facilitators received the standard three-day Incredible Years training.” The training included: 
co Training on the Incredible Years collaborative model and the Incredible Years 
curriculum; 
o Training on how to be a facilitator, which includes how to role play and how to help the 
parents and the interactive children work on techniques learned during class; and 
o Training on INTERFACE, which included information on how to conduct the first 
session and how to incorporate cultural components and sensitivity throughout the 
program. 
e The program manager met with facilitators before the start of any new group to help them prepare 
(to make sure they were ready to cover the curriculum, do role plays, discuss vignettes, and so 
forth for each meeting). 
e The program manager observed during the third or fourth session of the group. 
e Facilitators completed a standardized fidelity checklist at the end of each session. 
e Parents completed a feedback survey at the end of each session. The program manager provided 
each facilitator a summary of the parent survey responses after each session. 


*1 The Incredible Years Attentive Parenting Program is a six-week program (see also Appendix B and 
http://incredibleyears.com/programs/parent/attentive-curriculum/). Typically, an additional week was added so that 
the INTERFACE cultural contextualization piece could be offered during the first week’s session. 

22 Two interviewees mentioned that to become a certified Incredible Years parent facilitator, the facilitator must lead 
a certain number of groups and submit videotapes of the groups. No information was provided as to whether all or 
only some facilitators had received this certification. 
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e The program manager reviewed the facilitator checklists and the weekly survey data to see if 
there were any discrepancies or concerns that needed to be discussed with facilitators. 

e The program manager stayed in contact with facilitators via email and phone throughout the 
program offering. 


The Incredible Years program suggests that parents call each other during the week to serve as “buddies” 
as they practice the techniques they are learning in the program; however, the facilitators said that it could 
be difficult to get the parents to call each other, so often the facilitator made the phone calls. Also, as 
mentioned above, Incredible Years does not require two facilitators, but Wilder has found that practice to 
be a successful strategy for maintaining fidelity. Plus, as one interviewee noted, with two facilitators, 
“vou have different point of views. Sometimes, you’re able to connect with a parent and somebody else 
maybe is more connected to one parent. So they can have closer connections with the facilitator.” 


Parents’ Satisfaction with the Program 


One aspect of fidelity includes assessing participants’ satisfaction with the program. To measure this, 
parents were administered the Incredible Years Parent Program Satisfaction Questionnaire (Attentive 
Parenting Program) at the end of the programming (during the last session). The survey was not modified 
to include any items specifically related to the cultural contextualization components, so no information is 
available regarding participants’ responses to the cultural contextualizations. The questionnaire includes 
questions about satisfaction with the overall program, teaching format, specific parenting techniques, 
group leader(s), the parent group, as well as three open-ended questions. Example questions and response 
options are: 

e “I feel that the approach used to strengthen my child’s social and emotional behaviors in this 
program is” (response options: very inappropriate, inappropriate, slightly inappropriate, neutral, 
slightly appropriate, appropriate, greatly appropriate); 

e “Demonstration of parenting skills through the use of video vignettes was” (response options: 
extremely useless, useless, slightly useless, neutral, somewhat useful, useful, extremely useful); 
and 

e “I feel the leader’s teaching was” (response options: very poor, poor, slightly below average, 
average, slightly above average, high, superior). 


Wilder provided CAREI with anonymous participant level data for each survey question. The percent of 
respondents selecting each response option was calculated for each survey question. In addition, the 
information provided in the open-ended questions was summarized. A summary of the survey findings is 
presented below for each cultural contextualization cohort. For more information, see Tables F1-F7 in 
Appendix F for the African American contextualization cohort and Tables G1-G7 in Appendix G for the 
Hmong cohort.”* 


Survey Summary for the African American Cultural Contextualization Cohort 


Survey data were available for three of the African American cultural contextualization groups. A total of 
12 participants completed the survey (although not every respondent answered every question). 


Overall Program. Most respondents reported that their child’s social and emotional, pre-academic, self- 
regulation, and problem solving skills had improved slightly to greatly (see Table Fl, Appendix F). No 


3 The percent of respondents answering each survey question is provided in the tables. Percent totals in the tables 
may be above or below 100% due to rounding error. 
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respondents indicated that their child’s skills had decreased. However, 25% to 33% of respondents said 
that their child’s skills remained at the same level as before they started the program. When asked to rate 
their overall feelings about their progress at using skills taught by the program, 42% of the respondents 
felt optimistic. Likewise, 42% of respondents indicated that the approach used by this program was 
greatly appropriate to strengthen their child’s social and emotional behaviors. The majority of respondents 
(75%) responded that they would recommend this program to a friend or relative. Most respondents 
(83%) said they felt slightly to very positive about achieving their program goal. 


Teaching Format. Eighty-four percent of respondents reported that the content of the information 
presented was somewhat to extremely useful (see Table F2). One respondent responded negatively, rating 
the content as useless. Responses indicated that there were mixed opinions about the utility of different 
program activities (e.g., video vignettes, group discussion, practice of play and coaching skills at home, 
reading chapters of Incredible Years texts, weekly handouts, and practice during group sessions). Some 
respondents rated five of the six activities as slightly useless or neutral. Despite that, the majority of 
respondents indicated that these activities were somewhat to extremely useful. Use of practice or role play 
during group sessions was rated lowest (42% selected slightly useless or neutral), and group discussion of 
parenting skills and weekly handouts were rated highest (84% rated these as somewhat useful, useful, or 
extremely useful). 


Specific Parenting Techniques. Respondents were asked to rate specific parenting techniques: attentive 
child-directed play, academic and persistence coaching, social coaching, emotional coaching, teaching 
children to problem solve through play, and helping the child to control his/her anger (see Table F3). The 
majority of respondents rated these techniques as useful or extremely useful. For each of the techniques, 
one respondent rated the specific technique as either slightly useless or neutral. When asked to rate this 
overall group of techniques, 83% of respondents rated them as useful or extremely useful. 


Evaluation of Parent Group Leaders. In the Incredible Years Parent Program Satisfaction Questionnaire 
for the Attentive Parenting Program, respondents are asked to rate each group leader individually. 
Because the purpose of this survey analysis was not to evaluate individual group leaders, but rather to 
gain a better understanding of how respondents perceived the group leaders, the information for all group 
leaders was combined (for this reason, there are a larger number of responses for each question in this 
section; see Table F4). No respondents rated the group leader on the negative end of the spectrum of 
response options. In fact, half of the respondents rated the leader’s teaching and preparation as superior; 
the other half rated the leader as average, slightly above average, or high. All respondents were satisfied 
or extremely satisfied with the leader’s interest in and concern about the respondent and his or her child, 
and all felt the leader was helpful or extremely helpful. 


Parent Group. Most respondents indicated that the parent group was supportive and that other group 
members displayed interest in the respondent and his or her child (see Tables F5 and F6). However, 80% 
of respondents said that they would not like to keep meeting as a parent group (respondents could select 
yes or no). Responses to the question, “How likely is it that you will continue meeting with one or more 
parents in your group?” were mixed; responses ranged from the lowest (highly unlikely) to highest (very 
likely) indicators on the spectrum of ratings. However, the majority of respondents (63%) said they were 
somewhat to very likely to meet with one or more group members again. 


Open-Ended Responses. Parents are also asked to respond to three open-ended questions (see Table F7 for 
a complete list of responses). Six people answered the question, “How could the program have been 
improved to help you more?” Two people said the program could have covered more topics, and two 
others said the program could have been longer. One person commented that the curriculum was 
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“ageressive and ridiculous,” but another person said “I don’t think the program can be improved. I 
already think highly of the program.” Seven people answered the question, “At this time, do you feel the 
need for further parenting assistance?” Five people answered affirmatively, and two people answered 
negatively. Seven people answered the question, “What do you see as the main benefit of the Attentive 
Parenting Program?” There were a variety of responses, including learning new parenting techniques, 
helping children learn specific skills, and learning other parent’s perspectives. 


Survey Summary for the Hmong American Cultural Contextualization Cohort 


Survey data were available for four Hmong cultural contextualization groups. A total of 20 participants 
completed the survey (although not every respondent answered every question). 


Overall Program. The majority of respondents reported that their child’s social and emotional, pre- 
academic, self-regulation, and problem solving skills had improved slightly to greatly (see Table G1, 
Appendix G). However, one respondent said that their child’s social and emotional, self-regulation, and 
problem solving skills had become slightly worse. Sixteen percent to 21% of respondents said that their 
child’s skills in these areas remained at the same level as before they started the program. When asked to 
rate their overall feelings about their progress at using skills taught by the program, most respondents 
(79%) felt slightly to very optimistic. Likewise, all respondents said that the approach used by this 
program was appropriate to strengthen their child’s social and emotional behaviors. All respondents 
indicated that they would recommend this program to a friend or relative. In addition, all respondents said 
they felt slightly to very positive about achieving their program goal. 


Teaching Format. None of the responses concerning the teaching format were negative (see Table G2). 
All respondents indicated that the content of the information presented was useful or extremely useful. 
Likewise, all respondents felt that demonstration of parenting skills through the use of video vignettes, 
group discussion of parenting skills, and practice of play and coaching skills at home was somewhat to 
extremely useful. Some respondents rated reading chapters from Incredible Years texts, weekly handouts, 
and use of practice or role play during group sessions as neutral. 


Specific Parenting Techniques. Respondents were asked to rate the following specific parenting 
techniques: attentive child-directed play, academic and persistence coaching, social coaching, emotional 
coaching, teaching children to problem solve through play, and helping the child to control his or her 
anger (see Table G3). The majority of respondents felt that these techniques were somewhat to extremely 
useful. When asked to rate this overall group of techniques, all respondents rated them as useful or 
extremely useful. 


Evaluation of Parent Group Leaders. Respondents to this survey are asked to rate each group leader 
individually; however, because the purpose of this survey analysis was not to evaluate individual group 
leaders, ratings for all group leaders were combined (for this reason, there are a larger number of 
responses for each question in this section; see Table G4). No respondents rated the teaching on the 
negative end of the spectrum of responses options. However, one respondent rated the leader’s 
preparation as slightly below average. Having said that, the majority of respondents rated the leader’s 
teaching and preparation as superior (50% and 57%, respectively). Ninety-seven percent of respondents 
were slightly to extremely satisfied with the leader’s interest and concern in the respondent and his or her 
child, and all respondents found the leader to be helpful or extremely helpful in general. 


Parent Group. All respondents indicated that the parent group was supportive and that other group 


members displayed interest in the respondent and his or her child (see Tables G5 and G6). However, 88% 
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of respondents said that they would not like to keep meeting as a parent group (respondents could select 
yes or no). Responses to the question, “How likely is it that you will continue meeting with one or more 
parents in your group” ranged from the lowest (highly unlikely) to highest (very likely) of the response 
options. However, the majority of respondents (67%) said they were somewhat to very likely to meet with 
one or more group members again. 


Open-Ended Responses. Parents are also asked to respond to three open-ended questions (see Table G7 
for a complete list of responses). Thirteen people answered the question, “How could the program have 
been improved to help you more?” Four people said that the program was helpful and did not need 
improvement. Two people expressed that the program should be longer or cover less content to give more 
time for parents to learn. Other singular responses included that the leader should be more prepared and 
allow more advance notice for events outside of class, the videos should be translated into Hmong, and 
that the program was focused too much on one-to-one parent and child interactions. Fourteen people 
answered the question, “At this time, do you feel the need for further parenting assistance?” Eleven 
people answered negatively. The remainder of respondents expressed that the program was helpful and 
they could use further opportunities for practice. Seventeen people answered the question, “What do you 
see as the main benefit of the Attentive Parenting Program?” There was a range of responses, including 
the usefulness of videos, assignments, and examples; bonding with children; knowing themselves as 
parents; and helping their children become better people. 


CULTURAL CONTEXTUALIZATION 


According to staff members, the cultural contextualization of the Incredible Years program occurred 
several ways, which will be discussed below. The information for this section was obtained through the 
analysis of meeting minutes, documents, and transcriptions of interviews with key staff members. 


Use of INTERFACE 


The INTERFACE resource was adopted as the framework for helping facilitators incorporate culture into 
the programming by establishing relationships with the parents. INTERFACE was designed as a two- 
session module that would precede Incredible Years programming, but once Wilder moved to the six- 
week Attentive Parenting Program, INTERFACE was reduced to one session. An internal document 
provides the following additional information about INTERFACE: 


INTERFACE involves the articulation of family narratives to identify parenting values and goals. 
Then, using motivational interviewing techniques, parents will clarify these values and goals... 
the group facilitator will present the [TY curriculum in a manner that matches the values, 
regulatory focus (i.e., promotion or prevention) and target (i.e., parent or child) of the parents.... 
Role induction techniques will be used to ensure parents understand their roles and 
responsibilities throughout the course of the [TY program. Finally, cultural adaptations will be 
made through contextualization and augmentation based upon the information garnered from the 
... engagement module. (Judy Ohm, personal communication and internal document, 1/21/16) 


The INTERFACE curriculum and training manual are under development, but Wilder incorporated 
aspects of it into the three-day Incredible Years facilitator training (as mentioned above). Facilitators used 
INTERFACE during the first week of programming. During this time, deeper connections are made with 
the parents. Facilitators discuss parenting issues with participants — how they were raised, their current 
parenting styles, their motivations as parents, their hopes for their children, family and cultural values, 
and so forth. One person commented that “the INTERFACE exchange is rich.” 
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These parent narratives discussed during the first session were designed to increase parent engagement 
and buy-in to the program. The information was also used as a reference point to continue to incorporate 
culture into each session. 


We only use one session, it’s called the engagement session, and we use parts of the 
INTERFACE in that session to have parents talk about their own narratives and how do they 
parent, how were they parented, kind of trying to get them into the groove of there’s no right or 
wrong parenting. We just try to help them open up a little bit about what parenting is for them. 
Then we introduce what Incredible Years is and what we can provide and how we could possibly 
fill in things that they would like to learn about. (Facilitator) 


Facilitators took notes throughout a number of Incredible Years parent sessions during the grant period. 
The notes were summarized to gain a better understanding of the cultural contextualization that occurred 
during programming (Yang, 2015). The findings show that “contextualization is not generalizable for all 
groups regardless of background but that there are themes and styles of parenting that come up more than 
once in a variety of groups regardless of composition” (Yang, p. 2). Common themes included: respect 
for adults, social and emotional skills, culture, physical discipline, traditions. Yang concluded by saying 
that “the amount of themes can attest to the fact that every group is different and contextualization should 
be fluid along with the dynamic of the groups composition” (p. 7). 


Program Delivery 


Facilitators were matched with the cultural group when possible. Thus, African American facilitators 
were hired to facilitate the African American cultural contextualization groups; Hmong facilitators who 
speak Hmong were hired to facilitate the Hmong contextualization groups. 


Speaking in the Hmong language allowed facilitators an opportunity to translate key terminology in ways 
that would be more understandable. One of the Hmong facilitators explained that although Incredible 
Years is considered culturally competent, the curriculum cannot always be translated directly and still be 
understood by people from different cultural groups. For example, she said the concepts of pretend and 
imaginative play are difficult to translate in a way that would be understood by people in the Hmong 
culture. 


I think something I would say in terms of contextualization is that the national Incredible Years 
curriculum considers it as culturally competent across the board. But we’ve found that it’s not 
really, especially with the Hmong community, because for instance if I was to translate a lot of 
the vignettes, the concepts they talk about in those vignettes sometimes are not culturally 
competent. If I spoke about star wars and fighting out in space, I can’t translate that directly into 
Hmong and have it make sense.... So what I do is kind of modify concepts, not even modify, but 
try to relate it to what the parent understands or what the community understands. (Facilitator) 


Thus, an important piece of the cultural contextualization was to use language and terminology that would 
be more meaningful and relevant to the cultural group. For example, a common metaphor used in the 
Incredible Years programming is the pyramid that depicts the parenting strategies that should be used 
most frequently at the bottom of the pyramid (e.g., play, praise and incentives, and effective limit setting) 
and the parenting techniques that should be used sparingly at the top of the pyramid (e.g., ignoring, 
distracting, redirecting, and time outs). The facilitators discussed how they used a house metaphor or a 
gardening metaphor instead for the cultural groups. They built in time to discuss how the pyramid model 
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may not be as relevant in their traditions and then provided another metaphor that might be more relevant 
but that still emphasized the Incredible Years model. 


Video Vignettes 


Video vignettes are used throughout the program. Facilitators discussed how the people depicted in the 
videos are not very diverse and the vignettes are in English. They felt that in order to do a cultural 
adaptation, the videos need to look more like the people in the group and be in the language of the culture. 
One facilitator said, “...let's get to the video, the all-white videos. You know I think if we're going to do a 
cultural course it would make sense that we had that cultural piece—you know that we see a Hmong 
family, we see a Hispanic family, we see a Somalian family, and we see an African American family. 
That would have been nice.” Thus, for the Hmong groups, facilitators at first would translate the clips as 
the video played. Because it was quite challenging to translate as the video played, a Wilder staff member 
translated the videos into Hmong and recorded the translations over the videos. The idea was that the 
recorded translation would play while the video played; however, the voice-over was not very successful. 
A facilitator stated, “...when you translate to Hmong, things are a little longer, so sometimes the video 
would end and then the voiceover continued because it’s not over yet. So it was a little confusing for the 
parents.” One facilitator believes it would be better to play the video in English or read the translation of 
the video in Hmong. Another said that it might be better to use more role play by the facilitators to depict 
the lessons from the videos. 


KEY STAFF MEMBERS’ PERSPECTIVES 


Several key staff members acknowledged that, throughout the subgrant period, the initiative had trouble 
meeting their goals related to the number of African American and Hmong cultural contextualization 
groups held and the total number of parents and interactive children who participated. Some interviewees 
attributed this, in part, to “misperceptions around the type of grant it was.” One interviewee reported that 
the programming team thought, “It was an implementation grant in our viewpoint versus evaluation,” 
which is a misconception about the SIF program. When the grant was awarded in April 2013, the Hmong 
contextualization had been offered once and the African American contextualization had not yet been 
developed; thus, Incredible Years contextualized programming was not offered until September 2013. 
One interviewee reported, “And so from April to September we were working on ‘we have to get people 
trained, we have to...” You know there were a lot of things that had to happen in order to for us to be 
ready and so that’s why we didn’t actually pull groups until September.” Interviewees also talked about 
other implementation issues that resulted in them getting behind in their target numbers, such as a winter 
of bad weather and class cancellations, space issues, and the need to hire and train new facilitators. 


Although they acknowledged the difficulty they had in reaching their target numbers, the interviewees felt 
that several positive outcomes emerged from the grant funding. One person said: 


I think that we did miss our mark obviously, our numbers weren’t there. I think we have a good 
experience to talk about how that happened.... I mean I’m sure it was somewhat disappointing 
from the United Way’s part because we weren’t making our numbers, but yet, we were having an 
impact, we do have an impact ... look at the things we can validate. Can we validate that we’re 
working in cultural communities? Yes. Can we validate that the parents are coming in with a high 
level of stress when they’re living in poverty? ... And to know that that piece of it is yes, I mean 
you know the answer is yes, even though the numbers are small. (Interviewee) 
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Although the impact evaluation findings did not show a significant change in parents or children, 
interviewees reported that they receive comments from parents about how they enjoyed the programming 
and felt that it was beneficial. Interviewees also described how they will see a parent who has gone 
through the program at Wilder or in the community and witness the parents saying or doing things that 
were modeled in the class. Another interviewee described how the people hired to be interpreters for the 
class also found it beneficial: “They would talk afterwards. They would say, ‘I’m going to use this at 
home.’ So [it was as if] they were going through the class.” Another interviewee said, “I feel like parents 
who go through it, they really say they like it and they benefited from that, so I think it's a great program 
for parents.” 


Second, having the subgrant “frankly provided a testing ground for taking a program that wasn’t specific 
to Hmong or African American and gave SPPN (Saint Paul Promise Neighborhood) sort of a testing 
ground to figure out kind of how to do that.” This included changing the recruitment strategies throughout 
the subgrant period. One interviewee noted, “...It really is looking at reaching across the community and 
figuring out the pieces that are effective with various cultures, with various agencies, different parents...” 
Early on, Incredible Years administration staff members were trying to coordinate the programming 
(including the cultural contextualization components) and recruit parents. Recruiting has included posting 
flyers in the community and attending school and community resource fairs. Recruiting has also included 
building relationships with organizations that could refer parents to Incredible Years, such as Head Start 
and schools. During the grant period, they hired a Hmong parent navigator, whose job title quickly 
changed to “Recruiter and Community Outreach Coordinator.” She is responsible for developing 
marketing strategies to promote community interest. One interviewee stated, “culturally-based 
recruitment strategies matter.” She is promoting partnerships with other organizations with similar 
missions that do not provide parent education but would like that as part of their programming. One 
interviewee noted that the key lesson learned here is that it takes time to learn about community needs and 
to build relationships with community partners. She noted the importance of gaining trust and respect in 
the communities, so that you become a trusted resource for providing services. For example, if you are 
asking an organization to refer clients to your program, “it’s long-term—building relationships and trust 
because we’re really asking for them to hand over their clients, and they want to make sure that it’s 
quality, that we’re going to take care of them, we’re not going to alienate them.” 


CONCLUSIONS 


Although the grantees struggled with recruiting participants, during the grant period they facilitated 25 
groups that served 557 people — 202 parents and 355 children. This shows that as they honed their 
recruitment strategies, they were able to deliver the programming to more parents and children. However, 
in order to increase participation numbers, the programming was offered to community groups rather than 
strictly African American or Hmong contextualization groups. While this resulted in a larger number of 
parents and children receiving the Incredible Years programming, it did not provide larger sample sizes 
for testing the hypothesis that culturally contextualized versions of the Incredible Years curriculum would 
improve outcomes for parents and children in the two cultural groups targeted for this intervention. 
Again, because the purpose of this evaluation was to study the cultural contextualizations of the 
Incredible Years parenting program specifically offered to African American or Hmong cultural groups, 
findings from only the cultural contextualization cohorts were reported here. 


For the African American cultural contextualization cohort, 13 pre/post scores on the PSI-SF and 8 
pre/post scores on the SDQ were included in the impact evaluation analysis. For the Hmong 
contextualization cohort, 26 pre/post scores on the PSI-SF and 19 pre/post scores on the SDQ were 
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included. Thus, there was not much power to detect a difference between pretest and posttest scores. 
Paired ¢ tests were computed for the Hmong cultural contextualization cohort, and no significant 
differences were found for either instrument on the total scale or any of the subscales. Thus, there is no 
evidence to suggest that there was any effect on the Hmong cohort. Although ¢ tests were not conducted 
for the African American cultural contextualization cohort, effect sizes were calculated. The effect sizes 
were large and in the desired direction in three areas, which suggests that further examination of the 
impact of the African American cultural contextualization is warranted. There was a large difference (in 
the desired direction) from pretest to posttest on the PSI-SF Total Stress score (Cohen’s d = 0.74), the 
PSI-SF Difficulty Child subcale (d = 0.95), and on the SDQ Prosocial subscale (d = 1.02). 


Several limitations in the dataset also hindered our ability to further examine the data. For example, there 
were several instances where a parent had more than one pretest, posttest, or both. Because it was often 
not clear in the dataset which pretest/posttest scores were administered first, no data for these parents 
were included in the analysis. There were also discrepancies in the data; for example, group begin/end 
dates in the dataset did not always match up with dates provided to CAREI by Wilder. No daily 
attendance information was provided, so even if the samples had been larger, there was no way to 
examine dosage effects. Finally, it would have been interesting to connect parents to children in the 
dataset in order to examine the connection between PSI-SF and SDQ scores or to connect parents’ 
satisfaction levels as measured by the survey with their scores on the PSI-SF. Recommendations for 
future data collection include: 
e Track daily attendance data; 
e =Track all groups in which a parent participates (dates, age groups, interactive child for each 
group); 
e Attach parent data to their interactive children’s data; 
e Enter the date an assessment was administered rather than the date it was entered into the 
database; and 
e Enter item level information as well as scores for each assessment to allow the sample to be 
checked against the normative sample for each instrument. 


Evaluating implementation was also challenging because evaluators had to rely on information provided 
by key staff members through meetings, documents, and interviews. Evaluators did not observe (either in- 
person or through videotape) weekly sessions in order to examine implementation fidelity to the 
Incredible Years curriculum or to see how the cultural contextualization occurred. The contextualization 
pieces were not documented in the form of a manual or curriculum materials. The record of the 
contextualization is from interviews with key staff members and an analysis of notes taken during 
sessions. The lack of specific information about the cultural contextualization also makes it hard to 
attribute any effect, if one had been detected, to the standard Incredible Years program or to the 
Incredible Years plus contextualization condition. Recommendations around implementation include: 
e Create a manual for how to culturally contextualize the Incredible Years classroom while 
maintaining fidelity to the national curriculum; 
e Clearly demonstrate how training on cultural contextualization occurs during the three-day 
Incredible Years facilitator training; and 
e Adapt the Incredible Years materials to incorporate aspects of the cultural contextualization: 
o Add items to the observer checklists that include ways in which facilitators incorporate 
cultural contextualizations into the session; 
o Create a rubric for the program manager to use when observing Incredible Years sessions 
that includes cultural contextualization pieces; 
o Add items to the parent surveys administered after each session and to the parent 
satisfaction survey administered at the end of programming that expressly ask parents 
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whether they felt that their culture had been taken into account or that the terminology 
was understandable. 


The purpose of the grant was to test whether an evidence-based program that did not necessarily have 
explicit cultural aspects could be modified to include more culturally appropriate pieces (e.g., 
terminology, video vignettes, examples) and then show the same or better outcomes for the participants. 
This question fits nicely with the theory of change put forth by the SPPN, which posits that “...culture is 
the missing piece of the puzzle in the education process.” SPPN believes that partner programs should 
either be culturally-based at conception or should be adapted to include a cultural lens. In proposing the 
cultural contextualization of the Incredible Years curriculum, the question was “what would families need 
to make this more culturally appropriate and to feel connected to it?” Facilitators were then trained on 
INTERFACE and how to include cultural contextualization in the programming. The purpose of this 
grant was to validate the theory; however, because of the way it unfolded, the number of people 
participating in the cultural contextualization groups was lower than hoped. Project staff felt that the 
majority of the grant period was then spent on “getting the numbers,” rather than focusing on how best to 
refine the Hmong cultural contextualization and to develop the African American version. However, in 
the process, Incredible Years staff members reported that they learned several lessons: (1) adapting an 
evidence-based program is time-consuming work; (2) it takes time to build trust and rapport in 
communities, especially when you want to offer something new and test it at the same time; (3) 
recruitment strategies need to also be culturally contextualized; and (4) it may be best to partner with 
organizations that already serve the cultural groups your service is targeting, but do not currently offer the 
programming you provide. The test will be how Wilder and SPPN use the lessons they learned from 
participating in this project to continue to infuse culture into their programming. 
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APPENDICES 


Appendix A: Background on the Development of the Hmong Cultural 
Contextualization 


Prior to receiving funding from the Social Innovation Fund, Wilder worked with University of Minnesota 
Professor Richard M. Lee and his colleagues to develop and research a culturally contextualized version 
of the Incredible Years program for Hmong families. The cultural contextualization was created using a 
respectful and collaborative process where parents who participated in the curriculum gave much input 
into and feedback concerning the process and the experience. Every topic in the curriculum was looked at 
through a Hmong cultural lens and contextualized appropriately. The process emphasized a conscious 
integration of community members’ needs and wants as well as their socioeconomic and cultural contexts 
(Bermudez Parsai, Gonzalez Castro, Marsiglia, Harthun, & Valdez, 2011). Additionally, as recommended 
by Lau (2006), Wilder utilized a dual approach to cultural contextualization. One component of the dual 
approach involved contextualizing content so that the contextualization accommodated the distinctive 
contextual factors related to the problem in the target Hmong community. The other component involved 
enhancing engagement such that contextualization increased engagement but did not undermine the 
outcomes obtained with the original intervention. 


A group of Hmong parents who completed Incredible Years served as members of an advisory group for 
the program. The input they provided helped ensure that the Incredible Years contextualization was 
especially responsive to the Hmong community. For instance, as a result of the advisory group’s 
recommendations, the curriculum was contextualized to accommodate the learning style of the Hmong 
parent participants. More specifically, rather than completing a topic during each session, Hmong parents 
advised that they preferred to start discussion of a topic in one session, have time to process it afterwards 
at home, and then revisit it with their thoughts and reactions in a subsequent session. Another example 
involved a change in symbols used in the curriculum. For example, the traditional curriculum uses a 
pyramid model that represents approaches to building parent/child relationships. The parent advisory 
group suggested that the image of a house (with a foundation, more rooms on the main floor, moving up 
to the attic) was more relevant and meaningful in their culture. 


Dr. Lee’s research highlighted three main themes and nine subthemes for the Hmong cultural 
contextualization of the Incredible Years program (Lee, Vu, & Lau, 2013). These themes reflect deeper 
structural changes that need to be taken into consideration when creating a culturally contextualized 
Incredible Years curriculum for Hmong families. These data-driven themes address: a) communication 
and learning styles and preferences of participants (e.g., narrative, indirect/vicarious learning, 
uninterrupted turn taking); b) Hmong cultural factors that inform parenting (e.g., multiple children and 
caregivers, cultural role expectations, traditional parenting and child development); and c) acculturative 
stresses that affect parenting (e.g., time demands, paradigm shift, parenting concerns). Parent feedback at 
the end of weekly sessions and at the end of the program was overwhelmingly positive in terms of 
delivery format and content. Parents also expressed a strong preference for a Hmong-only group format 
and recognized the need to target less acculturated, Hmong-language-only families whose need for parent 
training might be especially great. Moreover, Hmong parents who had participated in Incredible Years 
indicated that they wanted to receive more parent training, and some former participants stated that they 
had encouraged other Hmong families to participate. 


4 This information about developing the Hmong cultural contextualization was initially provided in the final, 
approved SIF evaluation plan. 
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Appendix B: Wilder’s Incredible Years Program Descriptions 


Parents and Babies Program (10 weeks). Babies ages 0-12 months participate with their parents in this 
group. Childcare is provided for siblings by staff trained in the Incredible Years curricula. The Parents 
and Babies Program is a six-part program focused on helping parents learn to observe and read their 
babies’ cues and learn ways to provide nurturing and responsive care including physical, tactile, and 
visual stimulation as well as verbal communication. The series includes: 

Part 1 - Getting to Know Your Baby (0-3 months) 

Part 2 - Babies as Intelligent Learners (3-6 months) 

Part 3 - Providing Physical, Tactile and Visual Stimulation 

Part 4 - Parents Learning to Read Babies' Minds 

Part 5 - Gaining Support 

Part 6 - Babies' Emerging Sense of Self (6-12 months) 


Parents and Toddlers Program (10 weeks). Parents of toddlers ages 1-3 years participate in this group. 
Childcare is provided for siblings. The Parents and Toddlers Program builds optimal parenting skills and 
consists of an eight-part program focused on strengthening positive and nurturing parenting skills. The 
toddler age child will join their parent(s) for “interaction time” at the end of each session. Each part 
builds on the previous. The series includes: 

Part 1 - Child-Directed Play Promotes Positive Relationships 

Part 2 - Promoting Toddler's Language with Child-Directed Coaching 

Part 3 - Social and Emotional Coaching 

Part 4 - The Art of Praise and Encouragement 

Part 5 - Spontaneous Incentives for Toddlers 

Part 6 - Handling Separations and Reunions 

Part 7 - Positive Discipline - Effective Limit Setting 

Part 8 - Positive Discipline - Handling Misbehavior 


Parents and Toddlers Attentive Parenting Program (6 weeks). Parents of toddlers ages 2-4 years 
participate in this group. Toddlers join their parents for “practice time” during each session. Childcare is 
provided for siblings. Similar to the 10-week group, this program builds optimal parenting skills and 
incorporates curriculum elements such as those described above that are focused on strengthening 
positive and nurturing parenting skills and building social, emotional, and developmental skills for school 
readiness. 


Early School Age (Preschool) Attentive Parenting Program (6 weeks). Parents of early school-age 
children (preschoolers) ages 3-5 years participate in this group. Children join their parents for “practice 
time” during each session. Childcare is provided for siblings. The program focuses on strengthening 
parenting skills and consists of components which build upon one another. The series includes: 

Part 1 - Strengthening Children's Social Skills, Emotional Regulation, and School Readiness 

Skills 

Part 2 - Using Praise and Incentives to Encourage Cooperative Behavior 

Part 3 - Positive Discipline - Rules, Routines, and Effective Limit Setting 

Part 4 - Positive Discipline - Handling Misbehavior 
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Appendix C: Incredible Years Parent Programs Cultural Contextualization 
Logic Model 


Wilder Incredible Years Parent Education 
(Adapted from The Incredible Years Parent Program developed by Carolyn Webster-Stratton) 


Program Modalities 
Specific strategies, 
methods, and techniques 
are used to accomplish the 


Proximal (Short-term) Outcomes 


Targeted outcomes that the program has been 
shown in research to impact immediately and 2-3 


Targeted Risk and Distal (Long-term) Outcomes 


Protective Factors 


Outcomes thatthe 


Component: 
lY BASIC 


Parent Program 
(Toddler, Preschool ) 


Goal: Enhance posi- 
tive parenting interac- 
tions, coaching & at- 
tachment with chil- 
dren and proactive 
discipline. 


Center for Applied Research and Educational Improvement 


program goals. 


Goal Setting & Self 
Monitoring 
Video Monitoring 
Role Play & 
Behavioral Practice 


Group Support, 
Discussion and 
Problem Solving 


Cognitive 
Emotiona- 
Regulation Training 


Take home Practice 
and Reading 


FamilyMeal& Day 
Care 
Parent-Teacher 
Collaboration 
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Risk Factors: Parent, Child 
and Family: 


° 
¢ 


¢ 
A 


° 


Parent personal/ 
interpersonal problems 
Ineffective parenting skills 
Low monitoring 
Numerous stressors 
(negative life events, socio- 
economic disadvantage, 
and isolation) 

Low parent involvement 
with schools/teachers 
Child biological factors 
(developmental delays, 
ADHD, cognitive, lan- 
guage) 


Protective Factors: Parent: 


° 


Positive, nurturing, devel- 
opmentally appropriate 
parenting skills 

Parent support for child 
social, emotional and aca- 
demic development 
Parent support networks 
Parent partnerships with 
teachers/schools 

Parent effective communi- 
cation skills, anger man- 
agement, problem solving 


¢ 


years following program completion . 


Improved Parenting and Interpersonal 
Skills: - 


Increased parent positive interactions 
with child (e.g. nurturing, praise, coach- 
ing, and attachment) 
Increasedeffective parental limit- 
setting, replacing spanking and harsh 
discipline with non-violent, proactive 
discipline -Increased monitoring of chil- 
dren, predictable routines & safety 
proofing households 

Reductions in parental depression & an- 
ger 

Increased parent support systems 
Increased positive family communication 
& problem solving & collaboration with 
teachers 


Improved child behavior at home: 


¢ 


¢ 


Reduced behavior problems in child in- 
teractions with parents 

Increases in emotional regulation, social 
competence, problem solving and com- 
pliance with parents 


Improved child behavior with peers: 


a 


° 


Increased social and emotional compe- 
tence with peers. 

Increased problem solving -reduced be- 
havior problems , affect, social compe- 
tence and compliance 


Program is designed toimpact 
atlong term follow-up during 
adolescence. One 10-year 
follow-up study is available. 


Reduced Youth 
Antisocial Behavior: 
Less aggressive, de- 
structive behavior & 
conduct disorders 
Less depression 
Less use of drug and 
alcohol use 
Less likely to drop out 
of school 
Less criminal activity 
Less pregnancy 


Logic Model created by the Evidence-based Prevention and Intervention Support Center (EPISCenter) at Penn State University and 
Carolyn Webster-Stratton, Professor, University of Washington 
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Appendix D: Impact Evaluation Findings for the African American Cultural 
Contextualization 


PSI-SF Total Stress. For the Total Stress score, a score greater than or equal to 86 corresponds to a 
“clinical” score and a score lower than 86 corresponds to a “normal” score. This cutoff is represented as a 
dashed line in Figure D1. Figure D1 also shows the PSI-SF Total Stress scores for each parent (light gray 
lines) from pretest to posttest (13 parents met the criteria for inclusion). The downward slope of the 
heavier line (i.e., the mean change) shows that scores went down after participation in the program. The 
mean pretest score was 74.5 (SD = 21.6) and the mean posttest score was 62.0 (SD = 16.4). Cohen’s d 
was ().74, a moderately large effect. 


Changes in classification (i.e., clinical or normal) also occurred from pretest to posttest: 
e 3 African American parents had PSI-SF Total Stress scores that were categorized as clinical at the 
pretest. Of these 3, 0 had clinical scores at the posttest and 3 had normal scores. 
e 10 African American parents had scores that were normal at the pretest. Of these 10, 1 had a 
clinical posttest score and 9 had normal scores. 
e Insummary, 3 African American parents improved in classification, 1 worsened, and 9 stayed the 
same. 


Figure D1. PSI-SF Total Stress (African American) 
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PSI-SF Parental Distress Subscale. Figure D2 shows results for the 13 parents who met the inclusion 
criteria on the Parental Distress subscale on the PSI-SF.”> The pretest mean was 28.2 (SD = 11.6) and the 
posttest mean was 23.7 (SD = 8.9). Cohen’s d was 0.56. 


The dashed line in Figure D2 represents the cut score for a classification of clinical (above the dashed 
line) or normal (below the line). For the Parental Distress subscale, a score greater than or equal to 33 
corresponded to a clinical score and a score lower than 33 corresponded to a normal score. The changes in 
classification (i.e., clinical or normal) from pretest to posttest were as follows: 

e 5 African American parents had PSI-SF Parental Distress scores that were categorized as clinical 
at the pretest. Of these, 2 continued to have clinical scores at the posttest, while 3 had normal 
scores. 

e 8 African American parents had pretest scores that were normal. Of these, 0 African American 
parents had clinical posttest scores, while 8 had normal scores. 

e Insummary, 3 African American parents improved in classification, 0 worsened, and 10 
remained the same. 


Figure D2. PSI-SF Parental Distress (African American) 
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25 Note that in some figures, the number of lines will not be the same as the number of scores represented, because 
some parents (PSI-SF) or children (SDQ) had overlapping pretest and posttest scores and trajectories. 
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PSI-SF Parental Child Dysfunctional Interaction Subscale. The mean pretest score on the Parental 
Child Dysfunctional Interaction subscale was 19.3 (SD = 6.1); the mean posttest score was 17.5 (SD = 
5.5) (see Figure D3). Cohen’s d was 0.26. 


For the Parent-Child Dysfunctional Interaction scale, a score greater than or equal to 26 corresponded to a 
clinical score and a score lower than 26 corresponded to a normal score. In terms of changes in 
classification category from pretest to posttest: 
e 2 parents had PSI-SF scores that were classified as clinical at the pretest. Of these, 0 continued to 
have clinical scores at the posttest, while 2 had normal scores. 
e 11 parents had scores that were normal at the pretest. Of these, 1 parent had a clinical posttest 
score, while 10 had normal scores. 
e To summarize, 2 African American parents improved in classification, 1 worsened, and 10 
remained the same. 


Figure D3. PSI-SF Parental Child Dysfunctional Interaction (African American) 
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PSI-SF Difficult Child Subscale. Figure D4 shows that, for the Difficult Child subscale, the pretest mean 


was 27.0 (SD = 9.1); the posttest mean was 20.8 (SD = 6.2). Cohen’s d was 0.95, which represents a large 
effect. 


For the Difficult Child subscale, a score greater than or equal to 33 corresponded to a clinical score and a 


score lower than 33 corresponded to a normal score. Again, there was not much change in classification 
category from pretest to posttest: 
e 


3 parents had PSI-SF Difficult Child subscale scores that were clinical at the pretest. Of these, 1 
continued to have a clinical score at posttest, while 2 had normal scores. 


10 parents had scores that were normal at the pretest. Of these, 0 had clinical posttest scores, 
while 10 had normal scores. 


In summary, 2 parents improved, 0 worsened, and 11 remained the same. 


Figure D4. PSI-SF Difficult Child (African American) 
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SDQ Total Difficulties Score. Figure D5 shows the scores on the SDQ for the 8 children with pretest and 
posttest SDQ scores (lightly shaded lines). The heavily shaded lines shows that the pretest mean was 11.9 
(SD = 6.2) and the posttest mean was 9.6 (SD = 3.8). Cohen’s d was 0.39. 


Total scores between 0 and 13 were considered normal, 14-16 were considered borderline, and 17-40 
were considered abnormal. The dashed line in Figure D5 represents the cutoff between an abnormal 
classification (above the dashed line) and a borderline or normal classification (below the line): 

e There were a total of 2 African American children with abnormal scores on the pretest. Both had 

normal posttest scores. 

e One child who was classified as borderline on the pretest had an abnormal posttest score. 

e There were a total of 5 children with a classification of normal at both pretest and posttest. 

e Insummary, 2 children improved their classification, 1 worsened, and 5 remained the same. 


Figure D5. SDQ Total Difficulties Score (African American) 
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SDQ Emotional Problems Subscale. For this scale (see Figure D6), the pretest mean was 2.6 (SD = 1.7) 
and the posttest mean 2.1 (SD = 1.4). Cohen’s d was 0.25. 


Emotional problem scores between 0 and 3 were considered normal, a score of 4 was considered 
borderline, and scores of 5-10 were considered abnormal. Classification changes from pretest to posttest 
were as follows (again, the dashed line represents the cutoff above which scores are classified as 
abnormal): 
e 1 child had an abnormal emotional problems score on the pretest, but a normal score on the 
posttest. 
e 2had borderline emotional problems scores on the pretest. Of these 2 children, 1 child had a 
normal posttest score and 1 had a borderline score. 
e 5 children had normal scores on both the pretest and the posttest. 
e Overall, 2 children improved and 6 remained the same. 


Figure D6. SDQ Emotional Problems Subscale (African American) 
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SDQ Conduct Problems Subscale. Error! Reference source not found.Figure D7 shows that the mean 
pretest score for the SDQ Conduct Problems subscale was 2.5 (SD = 2.6) and the mean of the posttest was 
1.8 (SD = 1.8). Cohen’s d was 0.41. 


Conduct Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 
borderline, and scores of 4-10 were considered abnormal. In terms of classification change from pretest to 
posttest: 

e 3 African American children had abnormal Conduct Problem scores on the pretest. Of these 3 
children, 1 had a normal posttest score, 1 had a borderline posttest score, and 1 had an abnormal 
score. 

e Nochildren had borderline scores on the pretest. 

e 5 children had normal scores on the pretest. Of these, 4 continued to have normal scores on the 
posttest and 1 had a borderline posttest score. 

e Overall, the classification category for 2 children improved, for 1 child worsened, and for 5 
children the category remained the same. 


Figure D7. SDQ Conduct Problems Subscale (African American) 
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SDQ Hyperactivity Subscale. For the SDQ Hyperactivity subscale (see Figure D8), the pretest mean 
was 4.4 (SD = 2.6); the posttest mean was 3.9 (SD = 2.2). Cohen’s d was 0.42. 


Hyperactivity scores between 0 and 5 were considered normal, a score of 6 was considered borderline, 


and scores of 7-10 were considered abnormal. In terms of classification change from pretest to posttest: 


e 2children had abnormal scores on the pretest. Of these 2 children, 1 had a borderline posttest 
score, and 1 had an abnormal posttest score. 

e 1 child had a borderline score on the pretest and a normal score on the posttest. 

e 5 children had normal scores on the pretest and normal scores on the posttest. 

e Insummary, 2 children showed an improvement in classification from pretest to posttest and 6 
remained the same. 


Figure D8. SDQ Hyperactivity Subscale (African American) 
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SDQ Peer Problems Subscale. On the Peer Problems Subscale for the African American cultural 
contextualization groups (see Figure D9), the mean of the pretest was 2.4 (SD = 1.4) and the mean of the 
posttest was 1.9 (SD = 1.1). Cohen’s d was 0.24. 


Peer Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 
borderline, and scores of 4-10 were considered abnormal. Of the 8 children who had pretest and posttest 
scores, the following classifications were observed: 
e 2children had abnormal scores on the pretest; both had normal posttest scores. 
e 2had borderline scores on the pretest. On the posttest, 1 child had a borderline score and 1 had a 
normal score. 
e 4 children had normal scores on the pretest. Of these, 2 had normal posttest scores and 2 had 
borderline posttest scores. 
e Insummary, 3 children improved, 2 worsened, and 3 remained the same. 


Figure D9. SDQ Peer Problems Subscale (African American) 
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SDQ Prosocial Subscale. In contrast to the rest of the SDQ subscales, a higher score is more desirable 
than a lower score for the Prosocial subscale. Thus, in Figure D10, the area below the dashed line 
signifies a classification of abnormal and the heavier line shows movement from a lower score to a higher 
score, which indicates improvement from pretest to posttest on this subscale. On the Prosocial subscale, 


the pretest mean was 7.0 (SD = 2.1) and the posttest mean was 8.6 (SD = 1.3). Cohen’s d was 1.02, which 
is quite large and suggests a strong effect. 


Prosocial scores between 6 and 10 were considered normal, a score of 5 was considered borderline, and 
scores of 0 - 4 were considered abnormal. In terms of classification changes from pretest to posttest: 
e Nochildren had abnormal scores on the pretest. 


3 children had borderline scores on the pretest; they all had normal posttest scores. 


5 children had normal scores on the pretest; all 5 had normal scores on the posttest. 
Overall, 3 children improved and 5 remained the same. 


Figure D10. SDQ Prosocial Subscale (African American) 
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Appendix E: Impact Evaluation Findings for the Hmong Cultural 
Contextualization 


PSI-SF Total Stress. Figure El shows the PSI-SF Total Stress scores for each Hmong parent (lighter 
lines) from pretest to posttest (26 parents met the criteria for inclusion). The heavier line shows the mean 
score at pretest and posttest. The downward slope of the heavier line shows that scores went down after 
participation in the program (which is what is desired). The mean pretest score was 78.5 (SD = 23.7) and 
the mean posttest score was 71.5 (SD = 21.4). A paired ¢ test was performed to determine if the difference 
was significant. There was no evidence of a significant change in Total Stress score from the pretest to the 
posttest (#(25) = 1.09, p = 0.28, Cohen’s d = 0.36). 


For the Total Stress score, a score greater than or equal to 86 corresponds to a “clinical” score and a score 
lower than 86 corresponds to a “normal” score. This cutoff is represented as a dashed line in Figure E1. 
Changes in classification (i.e., clinical or normal) also occurred from pretest to posttest: 
e 9 Hmong parents had PSI-SF Total Stress scores that were categorized as clinical at the pretest. 
Of these 9, 3 continued to have clinical scores at the posttest and 6 had normal scores. 
e 17 Hmong parents had scores that were normal at the pretest. Of these 17, 2 Hmong parents had 
clinical scores at the posttest and 15 had normal scores. 
e In summary, 6 Hmong parents improved in classification, 2 worsened, and 18 stayed the same. 


Figure E1. PSI-SF Total Stress (Hmong) 
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PSI-SF Parental Distress Subscale. Figure E2 shows results for the 26 parents who met the inclusion 
criteria on the Parental Distress subscale on the PSI-SF. The pretest mean was 26.8 (SD = 8.9) and the 
posttest mean was 24.8 (SD = 10.5). There was no evidence of change from pretest to posttest for the 
Hmong group on the Parental Distress subscale (#(25) = 0.80, p = 0.43, Cohen’s d = 0.22). 


The dashed line in Figure E2 represents the cut score for a classification of clinical (above the dashed 
line) or normal (below the line). For the Parental Distress scale, a score greater than or equal to 33 


corresponded to a clinical score and a score lower than 33 corresponded to a normal score. The changes in 


classification (i.e., clinical or normal) from pretest to posttest were as follows: 

e 8 Hmong parents had Parental Distress subscale scores that were categorized as clinical at the 
pretest. Of these 8 Hmong parents, 3 continued to have clinical scores at the posttest and 5 had 
normal posttest scores. 

e 18 Hmong parents had pretest scores that were in the normal range. Of these, 3 had clinical 
posttest scores and 15 had normal posttest scores. 


e In summary, 5 Hmong parents improved in classification, 3 worsened, and 18 remained the same. 


Figure E2. PSI-SF Parental Distress (Hmong) 
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PSI-SF Parental Child Dysfunctional Interaction Subscale. For Hmong parents, the mean pretest score 
on the Parental Child Dysfunctional Interaction subscale was 24.3 (SD = 8.7); the mean posttest score was 
21.5 (SD = 6.6) (see Figure E3). There was no evidence of change from pretest to posttest for the Hmong 

group on the Parental Child Dysfunctional Interaction subscale (#(25) = 1.29, p = 0.21, Cohen’s d= 0.42). 


For the Parent-Child Dysfunctional Interaction scale, a score greater than or equal to 26 corresponded to a 
clinical score and a score lower than 26 corresponded to a normal score. In terms of changes in 
classification category from pretest to posttest: 

e 10 Hmong parents had PSI-SF Parental Child Dysfunctional Interaction subscale scores that were 
classified as clinical at the pretest. Of these 10 parents, 4 stayed in the clinical range at posttest, 
while 6 moved into the normal range. 

e 16 Hmong parents had pretest scores that were normal. Of these, 2 had clinical posttest scores and 
14 had normal scores. 

e In summary, 6 Hmong parents improved in their classification from pretest to posttest, 2 
worsened, and 18 stayed the same. 


Figure E3. PSI-SF Parental Child Dysfunctional Interaction (Hmong) 
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PSI-SF Difficult Child Subscale. Figure E4 shows that, for the Difficult Child subscale, the mean of the 
pretest was 27.4 (SD = 9.2) and the mean of the posttest was 25.3 (SD = 7.3). There was no evidence of 


change from pretest to posttest for the Hmong group on the Difficult Child subscale (#(25) = 0.98, p = 
0.34, Cohen’s d= 0.29). 


For the Difficult Child subscale, a score greater than or equal to 33 corresponded to a clinical score and a 


score lower than 33 corresponded to a normal score. Again, there was not much change in classification 
category from pretest to posttest: 


e 8 Hmong parents had scores classified as clinical at pretest. Of these 8, 4 continued to have 
clinical scores at the posttest, while 4 had normal scores. 


18 Hmong parents had scores that were normal at the pretest. Of these 18, none had posttest 
scores in the clinical range and 18 remained in the normal range. 


In summary, 4 Hmong parents improved, 0 worsened, and 22 stayed the same. 


Figure E4. PSI-SF Difficult Child (Hmong) 
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SDQ Total Difficulties Score. Figure E5 shows the scores on the SDQ for the 19 children with pretest 
and posttest SDQ scores (lightly shaded lines). The heavily shaded line shows that the pretest mean was 
10.1 (SD = 5.5) and the posttest mean was 7.9 (SD = 6.2). There was no evidence of change from pretest 
to posttest for the Hmong group on the SDQ Total Difficulties score (#118) = 1.7, p = 0.11, Cohen’s d= 


0.57. 


Total Difficulties scores between 0 and 13 were considered normal, 14-16 were considered borderline, 
and 17-40 were considered abnormal. The dashed line in Figure D5 represents the cutoff between an 
abnormal classification (above the dashed line) and a borderline or normal classification (below the line): 


Figure ES. SDQ Total Difficulties Score (Hmong) 
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SDQ Emotional Problems Subscale. For this scale (see Figure E6), the pretest mean was 1.9 (SD = 1.9) 
and the posttest mean 1.3 (SD = 1.6). There was no evidence of change from pretest to posttest for the 
Hmong group on the Emotional Problems subscale (t(18) = 1.64, p = 0.12, Cohen’s d = 0.39). 


Scores on the Emotional Problems subscale between 0 and 3 were considered normal, a score of 4 was 
considered borderline, and scores of 5-10 were considered abnormal. Classification changes from pretest 
to posttest were as follows (again, the dashed line represents the cutoff above which scores are classified 
as abnormal): 

e 2Hmong children had an abnormal score on the pretest and both had borderline scores on the 

posttest. 

e 1 child had a normal score on the pretest and a borderline score on the posttest. 

e 16 Hmong children had normal scores on both the pretest and the posttest. 

e Overall, 2 Hmong children improved, 1 worsened, and 16 stayed the same. 


Figure E6. SDQ Emotional Problems Subscale (Hmong) 
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SDQ Conduct Problems Subscale. Figure E7 shows that the mean pretest score for the SDQ Conduct 
Problems subscale was 2.2 (SD = 1.6) and the mean posttest score was 1.8 (SD = 1.8). There was no 
evidence of change from pretest to posttest for the Hmong group on the Conduct Problems subscale (t(18) 
= 0.88, p = 0.39, Cohen’s d= 0.35). 


Conduct Problems subscale scores between 0 and 2 were considered normal, a score of 3 was considered 
borderline, and scores of 4-10 were considered abnormal. In terms of classification change from pretest to 
posttest: 

e 3 Hmong children had abnormal Conduct Problems subscale scores on the pretest. Of these 3 
children, 1 had a normal posttest score, 1 had a borderline posttest score, and 1 had an abnormal 
posttest score. 

e 6Hmong children had borderline scores on the pretest. Of these, 2 had normal posttest scores, 2 
had borderline posttest scores, and 2 had abnormal posttest scores. 

e 10 Hmong children had normal pretest and posttest scores. 

e To summarize, 4 Hmong children improved in their classifications, 2 worsened, and 13 remained 
the same. 


Figure E7. SDQ Conduct Problems Subscale (Hmong) 
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SDQ Hyperactivity Subscale. For the SDQ Hyperactivity subscale (see Figure E8), the pretest mean was 
3.9 (SD = 2.6); the posttest mean was 3.3 (SD = 2.7). There was no evidence of change from pretest to 
posttest for the Hmong group on the Hyperactivity subscale (#(18) = 1.01, p = 0.32, Cohen’s d = 0.27). 


Hyperactivity subscale scores between 0 and 5 were considered normal, a score of 6 was considered 
borderline, and scores of 7-10 were considered abnormal. In terms of classification change from pretest to 
posttest: 


e 3 Hmong children had abnormal scores on the pretest. Of these 3 children, 1 had a normal 
posttest score and 2 had abnormal posttest scores. 


e 2Hmong children had borderline scores on the pretest. At posttest, 1 had a normal score and 1 
had a borderline score. 


e 14 Hmong children had normal pretest scores. Of these 14 children, 1 had a borderline posttest 
score and the other 13 had normal scores. 


e Insummary, 2 Hmong children improved in their classifications, 1 worsened, and 16 remained 
the same. 


Figure E8. SDQ Hyperactivity Subscale (Hmong) 
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SDQ Peer Problems Subscale. For the Peer Problems subscale (see Figure E9), the pretest mean was 2.1 
(SD = 1.4); the posttest mean was 1.6 (SD = 1.8). There was no evidence of change from pretest to 
posttest for the Hmong group on the Peer Problems subscale (#(18) = 1.07, p = 0.30, Cohen’s d = 0.34). 


Scores on the Peer Problems subscale between 0 and 2 were considered normal, a score of 3 was 
considered borderline, and scores of 4-10 were considered abnormal. Of the 19 children who had pretest 
and posttest scores, the following classifications were observed: 

e 3 Hmong children had abnormal scores on the pretest and posttest. 

e 4Hmong children had borderline scores on the pretest. All 4 had normal scores on the posttest. 

e 12 Hmong children had normal pretest scores. Of these 12 children, 2 had borderline posttest 

scores and the rest (10) had normal scores. 
e In summary, 4 Hmong children improved, 2 worsened, and 13 remained the same. 


Figure E9. SDQ Peer Problems Subscale (Hmong) 
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SDQ Prosocial Subscale. In contrast to the rest of the SDQ subscales, a higher score is more desirable 
than a lower score for the Prosocial subscale. Thus, in Figure E10, the area below the dashed line signifies 
a classification of abnormal. In this case, if the treatment has a positive effect, the pretest mean should be 
lower than the posttest mean (and the line should slope up from pretest to posttest). However, for the 
Hmong group, the pretest mean was 7.7 (SD = 2.0), which was higher than the posttest mean (6.3; SD = 
3.8). This change was not significant (7(18) = 1.38, p = 0.19, Cohen’s d = 0.34). 


Prosocial subscale scores between 6 and 10 were considered normal, a score of 5 was considered 
borderline, and scores of 0-4 were considered abnormal. In terms of classification changes from pretest to 
posttest: 
e 1 Hmong child had an abnormal score on the pretest. This child had a normal score on the 
posttest. 
e 2Hmong children had borderline scores on the pretest. Of these 2 children, 1 child had a normal 
score on the posttest and 1 had an abnormal score on the posttest. 
e 16 Hmong children had normal pretest scores. Of these 16 children, 4 had abnormal posttest 
scores and the rest (12) had normal scores. 
e Overall, 2 Hmong children improved, 5 worsened, and 12 remained the same. 


Figure E10. SDQ Prosocial Subscale (Hmong) 
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Appendix F: Survey Results for the African American Cultural 
Contextualization 


Table Fl. The Overall Program (African American) 


Considerably 
worse 


Worse 


Slightly worse 


The same 


Slightly 
improved 


Improved 


Greatly 
improved 


My child's social 
and emotional 
skills are (n=12) 


25% 


50% 


17% 


8% 


My child's pre- 
academic skills for 
language, reading, 
and persistence at a 
task are (n=12) 


33% 


33% 


8% 


25% 


My child's self- 
regulation and 
problem solving 
skills are (n=12) 


25% 


42% 


17% 


17% 


Very 
pessimistic 


Pessimistic 


Slightly 
pessimistic 


Neutral 


Slightly 
optimistic 


Optimistic 


Very 
optimistic 


My overall 
feelings about my 
personal progress 
at using the 
attentive and 
coaching parent 
skills are that I am 
(n=12) 


25% 


33% 


42% 


Very 
inappropriate 


Inappropriate 


Slightly 
inappropriate 


Neutral 


Slightly 
appropriate 


Appropriate 


Greatly 
appropriate 


I feel that the 
approach used to 
strengthen my 
child's social and 
emotional 
behaviors in this 
program is (n=12) 


17% 


17% 


25% 


42% 


Strongly not 
recommend 


Not 
recommend 


Slightly not 
recommend 


Neutral 


Slightly 
recommend 


Recommend 


Strongly 
recommend 


Would you 
recommend the 
program to a friend 
or relative? (n=12) 


25% 


25% 


50% 


Very negative 


Negative 


Slightly 
negative 


Neutral 


Slightly 
positive 


Positive 


Very positive 


My overall feeling 
about achieving 
my goal in this 
program for my 
child and family is 
(n=12) 


8% 


8% 


8% 


25% 


50% 
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Table F2. Teaching Format (African American) 


Extremely {seléss Slightly Neutral Somewhat Useful Extremely 

useless useless useful useful 
Content of information ; 8% ; 8% 17% 42% 5% 
presented was (n=12) 
Demonstration of parenting 
skills through the use of - - 17% 8% 8% 42% 25% 
video vignettes was (n=12) 
Group discussion of 
parenting skills was (n=12) ; ; ; ee ie ee oe 
Practice of play and 
coaching skills at home - - 17% - 8% 42% 33% 
with your child was (n=12) 
Reading chapters from the 
Incredible Years or 
Incredible Toddlers book : : cle oe ae eG ae 
was (n=11) 
Weekly handouts (e.g., tips 
& others) were (n=12) : z oe Be We ae oo 
Use of practice or role 
plays during group - - 17% 25% - 33% 25% 
sessions were (n=12) 

Table F3. Specific Parenting Techniques (African American) 
Extremely Useless Slightly Nciteal Somewhat Useful Extremely 

useless useless useful useful 
Attentive Child-Directed 
Play (n=12) - - 8% 8% - 42% 42% 
Academic and Persistence 
Coaching (1=12) - - 8% 8% - 42% 42% 
Social Coaching (n=12) - - 8% 8% - 50% 33% 
Emotional Coaching - - 8% 8% : 50% 33% 
(n=12) 
Teaching Children to 
Problem Solve through - - 8% 8% - 42% 42% 
Play (n=12) 
Helping child control 
his/her anger (n=12) : be as ¢ ue oe 
This Overall Group of 
Techniques (n=12) : ; pe ae ” ee aa 
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Table F4. Evaluation of Parent Group Leaders (African American) 


Slightly Slightly 
Very poor Poor below Average above High Superior 
average average 
I feel the leader's teaching : : : 9% 14% 27% 50% 
was (n=22) 
The leader's preparation : : ; 18% 5% 27% 50% 
was (n=22) 
Extremely rer Slightly Slightly : Extremely 
dissatisfied Dissiisiied dissatisfied ipateamien satisfied pauenet satisfied 
Concerning the leader's 
interest and concern in 
me and my child, it [sic] 7 ” ” ” 7 ae ook 
was (n=20) 
Extremely Slightly Slightly Extremely 
unhelpful Unhepe unhelpful Neutral helpful Hepa helpful 
At this point, I feel the 
leader in the program was - - - - - 35% 65% 
(n=20) 
Table F5. Parent Group (African American) 
ely - Unsupportive momen pat Neutral pomes tat Supportive ae 
unsupportive unsupportive supportive supportive 
I feel the group was - 7 - 9% 18% 21% 46% 
(n=11) 
Very ‘ Somewhat Somewhat Very 
uninterested Danteneen uninterested Neutra interested materesed interested 
Concerning other group 
members’ interest in me 
and my child, I felt they : : : a0 ie 4 i 
were (n=10) 
Highly ; Somewhat Somewhat , : 
unlikely Unlikely unlikely Neutral likely Likely Very likely 
How likely is it that you 
will continue meeting 
with one or more of the 18% - 9% 9% 9% 27% 27% 
parents in your group? 
(n=11) 
Table F6. Parent Group (Continued; African American) 
Item Yes No 
I would like to keep meeting as a group (n=10) 20% 80% 
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Table F7. Your Opinion (African American) 


Item 


Open-Ended Responses* 


How could the program 
have been improved to 


Cover more emotional topics 


e More topics 
help you more? (8 e =The curriculum was aggressive and ridiculous 
people commented; 1 e Get the book. More sessions—ours was only 7 sessions 
said N/A; | said No e It could have been a little more lengthy 
comment.) e I don’t think the program can be improved. I already think highly of the program. 
At this time do you feel e Could always learn more 
the need for further e Need to keep practicing the coaching methods we were taught 
parenting assistance? e Same 
Please elaborate. (9 e Yes because I am still a young, learning parent and I get frustrated 
people commented; 1 e Yes, other aspects of parenting 
said N/A; 1 said No > NG 
comment.) e No 
What did you see as the e <Achance to have dedicated time to think about my parenting styles and technique. 
main benefit of the e Everything was very helpful 
Attentive Parenting e Helping my children learn problem-solving skills. Helping my child learn. Prosocial 
Program? (9 people behaviors/skills 
commented; | said N/A; e Learning new coaching techniques 
1 said No comment.) e Other parent perspectives, professional input 
e ©The different parenting techniques when the child is at play. 


The thermometer 


* Minor corrections were made to enhance readability, but the intent of the statement is preserved. 
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Appendix G: Survey Results for the Hmong Cultural Contextualization 


Table G1. The Overall Program (Hmong) 


Considerably 
worse 


Slightly 
improved 


Greatly 


Worse Slightly worse The same ‘ 
improved 


Improved 


My child's social 
and emotional - - 5% 16% 21% 42% 16% 
skills are (n=19) 


My child's pre- 
academic skills 
for language, 
reading, and 
persistence at a 
task are (n=19) 


z Z - 16% 32% 32% 21% 


My child's self- 
regulation and 
problem solving 
skills are (n=19) 


- - 5% 21% 37% 21% 16% 


Very 


veo Pessimistic SueUy Neutral cae optimistic 


pessimistic pessimistic optimistic Optimistic 


My overall 
feelings about my 
personal progress 
at using the 
attentive and 
coaching parent 
skills are that I am 
(n=19) 


é - - 21% 11% 42% 26% 


Very 
inappropriate 


Greatly 


Slightly Neutral Slightly ie 


Inappropriate | inappropriate appropriate 


Appropriate 


I feel that the 
approach used to 
strengthen my 
child's social and - - - - 5% 53% 42% 
emotional 
behaviors in this 
program is (n=19) 


Strongly not Not Slightly not Neutral Slightly Recommend Strongly 
recommend recommend recommend recommend recommend 


Would you 
recommend the 
program to a - - - - 5% 32% 63% 
friend or relative? 
(n=19) 


Slightly Neal Slightly 


Very negative Negative negative positive 


Positive Very positive 


My overall feeling 
about achieving 
my goal in this 
program for my 
child and family 
is (n=19) 


E 2 - - 11% 47% 42% 
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Table G2. Teaching Format (Hmong) 


Extremely seless Slightly Neutral Somewhat Useful Extremely 

useless useless useful useful 
Content of information 
presented was (n=19) ” ; ” : ; nee pee 
Demonstration of parenting 
skills through the use of - - - - 11% 42% 47% 
video vignettes was (n=19) 
Group discussion of 
parenting skills was (n=19) : ; : ; ; kG One 
Practice of play and 
coaching skills at home - - - - 11% 32% 58% 
with your child was (n=19) 
Reading chapters from the 
Incredible Years or 
Incredible Toddlers book : : : ae one aoe ane 
was (n=9) 
Weekly handouts (e.g., tips 
& others) were (n=19) ; ; : oe 107 an ee 
Use of practice or role 
plays during group - - - 11% 16% 37% 37% 
sessions were (n=19) 

Table G3. Specific Parenting Techniques (Hmong) 
Extremely Useless Slightly Neneral Somewhat Useful Extremely 

useless useless useful useful 
Attentive Child-Directed 
Play (n=19) - - - - - 47% 53% 
Academic and Persistence 
Coaching (n=19) ; ” 7 i ” sie Paz 
Social Coaching (n=19) - - - - 5% 32% 63% 
Emotional Coaching r 7 7 7 11% 16% 74% 
(n=19) 
Teaching Children to 
Problem Solve through - - - - 5% 37% 58% 
Play (n=19) 
Helping child control 
his/her anger (n=19) ; ” ; ” he ne sha 
This Overall Group of 
Techniques (n=19) : : ” ; ; ae dee 

Center for Applied Research and Educational Improvement 
University of Minnesota 66 


Table G4. Evaluation of Parent Group Leaders (Hmong) 


Slightly Slightly 
Very poor Poor below Average above High Superior 
average average 
I feel the leader's teaching ; : : 5% 8% 37% 50% 
was (n=38) 
The leader's preparation : ; 3% : 11% 30% 57% 
was (n=37) 
Extremely er Slightly Slightly : Extremely 
dissatisfied Desai’ dissatisfied Ne Satisfied Sinai satisfied 
Concerning the leader's 
interest and concern in me : - - 3% 5% 1% 65% 
and my child, it [sic] was 
(n=37) 
Extremely Slightly Slightly Extremely 
unhelpful Dineen unhelpful Newtal helpful Hepa helpful 
At this point, I feel the 
leader in the program was - - - - - 35% 65% 
(n=37) 
Table G5. Parent Group (Hmong) 
mete - Unsupportive pomes iat Neutral pomer tat Supportive Mahe 
unsupportive unsupportive supportive supportive 
I feel the group was ; ; 5 ; 6% 50% 44% 
(n=18) 
Very A Somewhat Somewhat Very 
uninterested nmnterestet uninterested esac interested pterenee interested 
Concerning other group 
members’ interest in me 
and my child, I felt they : : : : ne one eee 
were (n=18) 
Highly F Somewhat Somewhat i F 
unlikely Unlikely unlikely Neutral likely Likely Very likely 
How likely is it that you 
will continue meeting 
with one or more of the 6% 6% - 22% 17% 17% 33% 
parents in your group? 
(n=18) 
Table G6. Parent Group (Continued; Hmong) 
Item Yes No 
I would like to keep meeting as a group (n=16) 13% 88% 
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Table G7. Your Opinion (Hmong) 


Item 


Open-Ended Responses* 


How could the program 
have been improved to 


Nothing much. 


e The program was really helpful. 
help you more? (13 e = Everything's good. 
people commented) e Overall the program itself was very helpful. It taught me to be more active in my baby sister’s 
early childhood as she grows up and is learning at the same time. 

e I would like it if the leader was more prepared and up to date with events outside of days that 
we meet in class. Don't wait for the last minute to tell us about outside events, or have to tell 
us about the events when the event ends on the day of. 

e | think if the videos were translated into the Hmong language it would be better for our 
group. 

e =I think if the program was broken out longer, so we can learn slowly, it would be more 
helpful, because I understand that there are some parents that take a longer time to learn all 
the information. 

e Less content and objectives to cover [each] session so there’s more time for parents to 
discuss, reflect and learn. 

e Maybe we should have more topics added to the class. 

e The program can improve with more families, however it’s understandable that families are 
busy, as time goes on there are less families showing up. 

e =I still think the program is geared too much on the | parent and 1 child ratio. Although this 
format may be useful in measuring results, it does not mirror what parents experience in the 
home with many children. 

e Let the kid and parent play as a group to see if they can see what the child likes in class 

e = Kuv xav thov kom hais lus hmoob (Translation: “I would like there to be more Hmong 
spoken/used.”’) 

At this time do you feel e =Nothing much. 

the need for further e None 

parenting assistance? e Notsure 

Please elaborate. (14 © No (4 people said “No”) 

people commented) e None at all, but it would be nice. The program was very helpful. 

e No, at this time, I think everything that was taught to us is good enough. 

e No. The concepts taught during incredible years were very good concepts. 

e Everything's good, helping one another being good parents. 

e The techniques that were taught over the last month were very insightful and very helpful. 


The handouts were very helpful at home and now it's work that needs to be, but the program 
was very helpful and the group leaders did a great job! 

Yes. I feel that I need more help with having to be able to tell my child to do something or 
stop doing something for me when being asked to do so. 

I think that if I had the opportunity to speak or share more on my challenges in the home with 
my children then I would be better able to utilize lessons from this program to help my 
family. 


What did you see as the 
main benefit of the 
Attentive Parenting 
Program? (17 people 
commented) 


I think the videos are a great way to teach, that way we have a visual to go off of. 

Hand out Assignments, videos, and examples were very beneficial. Everything else, and 
learning about your children's behavior was a good thing to know and how to resolve it. 
Maybe to understand the child's emotions. 

The main benefit is to let my child do things on her own and make her own decisions. 

Not only did it teach me to be a better role model for my sister, it also taught me to be more 
active and bond with her more. And I realize that the handout/ tips very helpful. 

Followings rules and bonding with child. 

Being able to understand my child more instead of just wanting things my way. 

Learning about how to know your child's emotion. Getting to know your child well. 
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Item 


Open-Ended Responses* 


Learning developmental milestones about my child's growth that I wouldn't otherwise have 
known. 

The main benefit of the program is as a parent we can lead our kids in a positive way. 

Help me be a better parent and teach my kids to be better kids 

The main benefit I got from this program was not to only raise a child, but so that I can 
understand and know myself as a parent. 

The program was very helpful and at the same it taught me to grow as well; while focusing 
on [child’s] needs too, to grow and develop at the same time. 

The main benefit was the ability to have others support me while I take the time to learn these 
lessons. Often times in the home, I feel that all the responsibility is on myself to [sic] my 
spouse. 

Very helpful 

Ntau yam. (Translation: “Many things.”’) 

Kev sib pab thiab sib raug zoo (Translation: “Sharing and relationship building”) 


*Minor corrections were made to enhance readability, but the intent of the statement is preserved. 
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